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ABSTPACT 

This book is a compilation of papers that consider 
the current state of the art of competency/performance-based teacher 
education. Organized into six parts, the first part of this book 
reports "ihree frames for conceptualizing research in competency-based 
education. The keynote address owtlines five major areas for 
research: assessment of professional competency, program 
effectiveness, effectiveness of instructional strategies, social 
context, and institutional change. Parts two through five focus on 
these areas of research. (Editor/PD) 
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University. 



BEST COPY AVAILABLE 



COMPETENCY ASSESSMENT. 
RESEARCH, AND EVALUATION 

A Report ot a 
National Conterencp 
University ot Houston 
March 12.15. I9r4 



SponscreO Dy 

The Multi-State ConsorTjijtM on Performance Based Teacher Education 
The National Commission on Performance Based Education 
The National Consortium of CBE Centers 

The National institute of Education. Program on Teaching and Curriculum 

Thr* Committee on Performance Based Teacher Education. 
Association of Colleges for Teacher Education 

Division of Education Systtms Development. United States Office of 
Educat'on 

Professional Development Center. Un.versitv of Houston 
Teacher Corps 



Copies of this report may be obtained 

American Association of Colleges 

for Teacher Education 
Suite 610 
1 Dupont Circle 
Washington. D C 20036 



S3 00 fronri 

National Dissemination Center for 
Performance Based Education 
150 Marshall Street 
Syracuse University 
Syracuse 

New York 13210 



This Conference was sponsored and this report was published as part of 
grants from the U S Office of Education and National Institute of Education. 
Department of Health. Education, and Welfare The opinions and other 
content do not necessarily reflect Ve position or policy of the Agency, and 
no official endorsement should be inferred 



ERIC 



OCST COPY AVAIl.flBlF 

CONTENTS 



FOREWORD 

PART I 

Chapter 1 

Chapter 2 

Chapter 3 

PART II 

Chapter 4 

Chapter 5 

Chapter 6 

Chapter 7 

Chapter 8 

Chapter 9 



COMPETENCY ASSESSMENT, 
RESEARCH. AND EVALUATION 

W Robert Houston. Conference Director 
and General Editor of the Report 

FRAMEWORK FOR RESEARCH 
AND DEVELOPMENT 
Conceptual Model of R & D for CBE 
Frederick J McDonald 
National institute of Education and 

Research m C/PBE 
Garry L. McDamels 
Closing the Knowledge Gap 
H Del Schalock 

ASSESSING PROFESSIONAL 

COMPETENCIES 

Listening to Different Drummers: 

Evaluating Alternative Instructional 

Models 
Bruce R Joyce 

Task Force on Relating Teacher 

Competence to Pupil Outcomes 
Donald M. Medley and David Krathwohl 
A Practical Approach to a Complex 
Problem 

George Gustafson and Marjone Powell 
Classroom Observation 
Robert S Soar 

inservice • Needs Assessment 

• Competency Based Teacher Education 

Gilbert F Shearron 

Assessing Teacher Competence 

Beatrice A. Ward. Greta Monne. and 
David Berliner 



Page 



11 



22 
34 



61 

82 

88 
100 

112 
124 



ERLC 



BEST COPY AVAIUBLf 



PART III 



^*0»,U 0 AutVt!'^* .riij n.U' ll Potior 



PART IV 



f-f f-f CTVFNtbS Of NSTRUlT'ONAI 
[ v i'ij.it rnj VV.ivs To H<>ip Tf.\u runs !o 



iv:b 
190 



PART V 

C raptor -d 



PART VI 

Cnapto* ly 

Chapter ?0 
C»^»iptef 

Cnapter 22 

AUTHORS 



ERIC 



soc:.Ai. lontext 

Tr«f Poiit ls o* Teacher Competence 
Harvey H Scrtbner and leonafd B Stevens 
The Dec'^^'OM MiiKif\g Process m Pitinnmg 
ar^d iniplemeniing EduLat'i)^ Pfugft^rns 
Ir^'it Meet tr\e Needs ot Our D»verst» 
S{>cietv 
Ca? A G'ant 

iNST.TuT:ONAL CHANGt 

SorT'.e U^ sLt^O^iHy V «ows ot*. if\sI«tuttoruil 

Cf><inqe 
Don Dav«es 

TtiSk Force on inst'tut>or\al Char-ge 

James Over and DeM Feider 

Mode! ior Developing a State Syf.tef^^ Uv 

Manaa»ng Performance Programs 
Uotx'tr C Butk^'or* 

impiementatior^ ot CBTE v«ewed as a 

Developmental Process 
Gene E Hail 



?0? 



216 



22 i 
232 

234 

252 
2V0 



BEST COPY AVAILABLE 



FOREWORD 



The sme qua non of competency/performance based programs is assess- 
ment, research, and evaluation, yet m these areas remam the knottiest 
problems faced by educators today They thrust educators beyond the 
cognitive into assessing the performance of teachers and other educational 
professionals They press for measures of effectiveness m assessing the 
coryseQjences of the professional s actions 

C/P8E provides the basis for developing assessment and evaluation 
techniques and employing them m research! efforts In a recent monograph, 
the Committee on Performance Based Education of AACTE noted that this 
movement offers an opportunity for significant research because (1) in- 
creased precision m defining goals, identifying assumptions, and measuring 
outcomes greatly improves the conditions for analyzing cause and ef*ect 
relationships within teacher preparation programs, and (2) the possibility 
of having groups of teachers with more clearly identified constellations Of 
abilities enhances the probability that relationships between teacher prep* 
aratton and pupit outcomes can be fruitfully studied ' * 

Schaiock. in proposing a research strategy as part of the efforts of the 
Consortium of CBE Centers, emphasized that. 'At present we have only the 
barest of data on program cost when operating withm the CBTE mode, and 
no evidence as yet upon cost/benefit relationships We also have no firm 
evidence that one set of competencies is more productive of learning m 
Children than another set. that one level of competency definition is any 
better than another level, that one approach to program organization and 
instruction is better than another in bringing about desired competencies, 
etc . etc Again, questions of this nature can be answered only through 
research, and at this pomt m time such research has not and is not being 
undertaken • Albert Shanker. in a column n the New Yarn Times, con- 
cluded that Instead of hard data and controlled experimentation, educa- 
tional panaceas are put forth on the basis of subjective impressions. " 

The importance of building this data base is acknowledged by all. withm 
recent years, an increased effort can be perceivel Many C/PBE projects 
have integrated research into program development. Research and De- 
velopment Centers are conducting series of studies, the National institute 
of Education has planned a series of conferences and activities to devise a 
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thematic research strategy rrmmtjership m the Artiencan Education Re- 
search Association has increased rapidly, and other organisations are em- 
phasising assessment and research m their programs 

CeMainly. these areas— assessment, research, and evaluation — are the 
most challenging m education today, and the most powerful catalysts to 
improved training of educational personnel To consider the current state of 
the art. and to encourage further efforts, a conference was organised to 
study CBTE The papers included herein were delivered at the National 
Conference on Competency Assessment Research and Eva luation 
held at the University of Houston on March 13. 14. 15. 1974 

The Conference was designed so that eminent educators and researchers 
could speculate on needed thrusts, consider current achievements, and 
interact on major issues More than 250 mvited participants attended. 

The Report is organised into six parts. The first part reports th'^«) frames 
tor conceptualising research m CBE Frederick McDonald ^d the 

keynote address m which he outlined five major areas for reo9 assess- 
ing professional competence, program effectiveness; effectiveness of 
instructional sttalegies. social context: and institutional change Each of 
these areas, m turn, wns the focus of a major presentation, several clinics, 
and a task force These presentations are included as chapters m the last 
five parts of the report, and are self-explanatory. 

Thanks are due to so many persons who made that conference and this 
publication possible The directors of projects and programs sponsoring the 
conference responded to numerous calls for assistance in identifying speak- 
ers, program personnel, participants, and m making fiscal and other ar- 
rangements These sponsors aie described m detail below. 

The Multi- State Consortium on Performance Based Teacher Education ts 
a federally funded project (Title V. ESEA; Teacher Corps. National Center 
tor the Improvement of Educational Systems) designed to assist the par- 
ticipating stat«»s (Washington. Oregon. Utah. New Jersey. Ansona. Minne- 
sota. Vermont. New York. Texas. Florida) m developing appropriate policies 
related to each state's unique approach to performance education. The 
Consortium also provides a national communication and dissemination 
resource . uough the publication of a monthly newsletter. PBTE, and a series 
of monographs Theodore Andrews of the New York State Education De- 
partment (the administering state) is Director of the project 

The Nationa! Commfssion on Performance Based Education seeks to 
improve programs for the preparation of educational and other professional 
personnel by initiating, supporting, and coordinating PBE research and 
development ac*ivities Direction for these activities will be provided through 
the implementation of a comprehensive Research and Development plan, 
with initial efforts focused on teacher preparation The Commission is sup- 
ported by a grant from the Rockefeller Brothers Foundation; Frederick J. 
McDonald. Educational Testing Service, is executive director 

The National Consortium of CBE Centers is an informal association of 
institutions involved in the development and implementation of Competency 
Based Teacher Education. At present, there are nine National CBE Centers' 
Florida State. Georgia. Houston. Michigan State. Syracuse. Teachers 
College-Columbia. Teaching Research/Oregon College of Education. To- 
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ledo and Wis<ons«n Iho Coiibortium serven as the coordinating body for 
the CDE Centers, strrvus as «i conceptual forum focusing upon the fefme* 
men! and advancement ol CBt concepts, provides a source of CBE leader^ 
ship at the national level, functions as a clearinghouse for providing develop* 
mental assistance to meol pnonty CBE needs, commissions papers and 
publications on unresolved issues, and sponsors thmK tanK* symposia on 
Key CBE topics Notman Dodl. Florida State University, is the current 
chairman of the board o* directors, and John M Hansen is the executive 
secretary 

The National Institute of EOucatton (NiE) was created m 1972 to support 
the policy of providing every person with equal opportunity to feceive a high 
qutilitv education Under the leadership of Thomas K Gionnan. Jr . the 
institute has established a number of priorities and programs to meet its 
goal 

The Program on Teaching and Curriculum is one of these ma|or efforts 
This Program s focus is to assist teachers m providing effective instruction 
to children in addition to ongoing work m research on teaching and cur- 
riculum renewal, the Prograni is developing a targeted research and devel- 
opment agenda to Qiiide future studies of teaching and curriculum 

The Committee on Performance- Based Taacfier Bducatwn of the Amer- 
ican Association of Colleges for Teacher Education gives direction to the 
Association s national project on PBTE Composed of represertatives of 
coUeges and universities, state education departments and the organi;red 
teaching profession, it seeks to clarify the PBTE concept, study the state 
of the art. stimulate dialogue about PBTE «; promise and limitatior^s. provide 
practical help to operators of PBTE programs, and make recommenda- 
tions which Will assist in achieving the potential of PBTE The major activt- 
lies of the project mchjde the publication of pertinent monographs and the 
sponsorship of various types of training opportunities The project is sup- 
ported by NCiES ot the U S Office of Education through the Texas Educa- 
tion Agency, and is directed b> Karl Massanart. AACTE 

The purp'^se of the Professt.tnai Development Center. University of Hous- 
ton. IS to facilitate the development of CBE programs for preparing profes^ 
sional personnel m Texas The Resources Center tn room 466 is part of the 
Center In additton. it is cosponsonng with the Texas Education Agency and 
the Multi-State Consortium a series of regional conferences and provides 
support services for program development The PDC is a special project of 
the Teacher Corps W Robert Houston is director of the PDC. Sarah White 
IS associate director of the PDC 

Teactier Corps is dedicated to improving the education of children from 
low income families It specifically includes authority to broaden programs 
of teacher preparation from prospective teachers, as well as those experi- 
enced teachers not in the schools Now preparing for its ninth 2-year cycle. 
Teacher Corps supports some 80 projects m 36 states. Puerto Rico, and the 
District of Columbia Approximately half of the projects are m major cities, 
and three-fourths are m elementary schools The heart of each project is 
the Teacher Corps team of interns and experienced teachers working to- 
gether with the schools, their communities, and their nearby universities m 
the planning and operating of innovative programs for the framing of teach- 
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ors Iht? (Jiiftctor ul t*».ti hi'i i uips is W»Hi.im Sm»tn .^nd the fissociiite di- 
rector IS Russell Woods 

bcvoriii meetings of the Coordinating Commtttee ot the National Com- 
mission un PBE were devoted to this topic Frederick McDonald Theodore 
Andrews Howard Coron Alien Schmieder David Potter Horace Aubef tine. 
Karl Massanan and RotK^rt Houston conceptualised the program design, 
suggested areas ot research to .t>e included and recommended a number 
of the speakers tor the conterence 

Garry McDameis and frank Sobol identified topics for sever*ii sessions 
and encourtiged many ot the participants to attend 

All of the organisations listed above and particularly their directors were 
mtluentiai m shaping the fmai program format, identifying educators who 
are concerned with research m CBE arid supporting the conference both 
tinanciaiiy and conceptually James Steffensen continually provided ideas 
and a resiiierM backboard for ideas during me development stages 

Thanks too must be accorded to program personnel Following the con- 
ference many of the speakers reviewed and edited their presentations on 
the basis ot quf»stions and feedback Task force synthesisers consolidated 
•.fver.ii hoiirs i»f d«si ussion into a 'ew pages and several relevant questions 
Their efforts heipeO make the conffjrence successful, while their careful 
atier^tton lo doia- har, made this report more useful 

Dean Robert Mowsam taciiitatod use of the College of Education at 
Houston and provided leadership m conference activities While more than 
faculty, staff and graduate assistants worked in the conference, the 
following deserve special identification as members of the arrangements 
committee and chairpersons of various committees Eugene Cbiappetta. 
registration Barry Peiton. pubncity. Joseph Schroeder and Jay Shores 
audio visual equ-pment and resources center. Ida Stewart, hospitality, and 
Bruce Thompson transportation 

Obtaining manuscripts from authors, copy editing V^em. and preparing 
them tor pr.nt<ng were accomphshGd by Barbara S Coleman and Sheila 
F(iiO University of Houston and Anne Fmch. New York State Education 
Department layout and format was designed by Pat Schrader and Parker 
Floyd of the university of Houston Prmtrng Plant ard the publication was 
printed by the Muiti- State Consortium 

Jur^e 1. 19/^ 
Houstcn Te»as 

W Robert Houston 

Conteience D«r<«cto' tinO General Editor of the Report 



roOTSiOTtS 



ERIC 



BEST COPY AVAILABLE 

PART I 



FRAMEWORK 

FOR 

RESEARCH AND 
DEVELOPMENT 



BEST COPY AVAIIApir 



1 Conceptual Model 
of R&D for CBE 



Pr*d«rtok J. McDonald 



fhere are two questions which are the toci tor analysis and discussion 
at this conference given that there is a definable set of events called 
compotencv Dased education, what constitutes evidence that it produces 
Its claimed effects and how much evidence is sufficient before we decide 
that It IS an effective strategy for training teachers'' When we bring together 
a gro^ip of people as diverse as this group, each brings his or her own ideas 
about What the significant research problems are m competency based 
education What we are trying to stimulate as we go through this conference 
IS. however, to focus your attention on those two questions 

in my op.non. the ^oa\ of our research and development efforts ought 
not to bo to prove that competency based education is effective That is 
an impossible task and can probably never be realised However, it the 
problem IS taken to be to establish what constitutes sufficient evidence so 
that competency based educatron may be established as a matter of social 
policy, then a research and development program is more readily conceived 

My role IS to lay out what » think are some of the major problems and the 
major questions which set the network of problems on which evidence is 
needed The iog»c of this presentation is to describe the problems for which 
evidence is needed i will be answering the question, what are the critical 
questions which must be answered if we ate to fmd whether or not com- 
petency based education makes a difference*^ In some cases, i will suggest 
the crocf .or agamst which the evidence ought to be projected The question 
of how much evidence is sufficient is »nore difficult to answer, and i will 
consequently have connderably lees to say about it 

The problem of what constitutes sufficient evidence that competency 
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based education (uujMt to Ue the style and strategy for training teachers is 
part logical aoA part psyc huluqical The logical aspect relates to the struc- 
ture, amount, and Kinds ot evidence needeo to constitute sufficient proof 
that the theory or model or strategy is useful. The psychological aspect 
reuves to what »nd«viduals will demand because of their ♦ntuitive and prefer- 
ential reactions to the concept The psychological aspects of belief and 
acceptance are not tclaHy r<^sponsive to the logic of data. 

One component of the psychological aspects of stimulating conviction is 
that as con^p.?tency based teacher education catches on. significant soctai 
implications for tht? teaching profession will be apparent. As a consequence, 
people Whose lives will be affected by competency based education are 
lookmy at the political implications of certifying teachers on the basis of 
demonstrated competence Their concern is twofold first, that the concepts 
of competence Will be defined independently of the conditions of teaching, 
second, that the criteria of competence will not take into account other 
factors that influence both teaching and learning. 

The conclusion to be drawn from this concern is that research designs 
must include the variety of factors that probably impinge on teaching per- 
tormances tind their con*iequences m learning It is reasonable to assume, 
tof example, that variations in administrative structure, curriculum, and 
LharacleribtiCb of children and teachers affect teaching performance itself 
and also learm^q it is ubwious that an important research problem m its own 
right »s to measure the relative effects of such factors Descriptions of 
competence divorced from the conditions of teaching and learning are not 
likely to be behevabie 

The criteria tor assessing the sufficiency of ev<dence appear to be of two 
kinds One set is related to the validity of the results ot the research -what 
data support a hypothesis that a teaching performance is effective, and h'^w 
qenerahzabie are the results The second set are more elusive, reiaied 
more to the cont<»xt and content of the research The research must be 
believable in that it attends to the complexities and realities of leachmg 
i address myself primarily to the kinds of knowledge needed to support 
the proposition that competency training is the most effective way to 
educate teachers i wm discuss several basic research problems whose 
solution IS required if the effectiveness of competency training is to be 
demonstrated and accepted Other speakers wtll discuss these problems m 
greater detail, elaborating their conceptual and methodological difficulties. 

Let me begin with what I thir^k is the basic problem, which hanpens also 
to be fraught with implications of the kind previously described: V at is. what 
«s the nature of teaching competence*? Some of us seem to assume that 
because we nave mo* * precise descriptions of teaching competencies that 
we have resolved the question of what is teaching competence. One of the 
most commonly stated accusations aimed at competency education is that 
we believe that a list of behavioral descriptions of competence constitutes a 
description of competence We are quite properly asked. How do you know 
that if the teacher performs as described that the performance will have a 
desirable effect on student learning'?" 

A competence may be described as a behavior of a teacher, for example. 
askz students to give reasons for t^.eir opinions on controversial topics 
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being discussed in class Pri?sumably. a high frequency of such state- 
ments indicates that the teacher has the behavioral performance in his or 
her repertoire But is such information evidence of competence"' It is only if 
we assume that the appearance of the behavioral act is synonymous with 
competence 

Such an approach is rightly charged, m my judgment, with indifference 
to the consequences of the event The more cymcal of cntics pomt out that 
this concept of competence leads logically to certifying as competent 
teachers who emit the behavioral performances in the absence of students 
While such criticisms are unnecessarily harsh, they pomt to the problem 
IS competence to be defined m terms of effects manifested in students, 
in terms of observable acts of teachers, or m terms of pairs cf acts and their 
effects'^ 

\X seems to me that we can no longer avoid saymy that teaching com- 
petence must be defined m terms of effects on pupil learning Unless the 
research and development movement behind competency based education 
addresses the problems implicit m that concept of competence, it seems to 
me there is no hope of ever persuading anybody that the competency leased 
education movement is doing anything other than developing semantic 
descriptions of what constitutes competence. I propose that the first and 
most important research problem to attack is the question of what teaching 
performances produce significant effects On learning 

There are two general research methodologies that may be used for this 
purpose t would like to describe them very briefly The first of these is an 
inductive approach to the analysis of the relationship between teaching 
performance and student learning. This methodology proceeds as follows. 
The research should be conducted so that actual teaching may be observed, 
preferably that of more e)iperienced teachers You need not have precon- 
ceptions about What kmos of performances are likely to affect student 
learning, or your observations of performance may be guided bv theory or 
previous research Put you must have a clear and specific conception of 
the kinds of student learning that you are going to study. 

The Simple form of the methodology consists of measuring student learn- 
ing at two points m time • Given two such measurements obtained on each 
of the students m the classes of a sample of teachers, a regression of the 
later or positest scores is performed on the earher or pretest scores The 
results of this statistical analysis can be diagramed as in Figure 1 1. 

The pomts a^e represented by the mean scores for a teacher's class, the 
first number being the mean for the pretest, the second, the mean for the 
posttest The line through the pomts »s called a regression line and points 
on It represent expected scores Given the information m the analysis, we 
expect T. s posttest mean score to fall on the line at T^e; similarly, we would 
expect T s posttest mean to be located on the line at the same point. Both 
posttest means, however, locate T. above the line and T below the line. 



•Le^vnmg .n r^.s speecn .s an ommbus category ^eant to include aii de&itabie changes «n 
students coqnttive attcct.ve and socai Nor am i concerned about the method o! measure- 
ment o» (Mi.t uit.es ot moasuf.nq some changes The reseat chef needs to be specific about 
me cr>;inges des<ted »ind h.ive d#»veiop«»d tehabie and vai»d ways ot measuung them 
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FtOur«1.1 ^ T. (45, 70> 




K T, (20. 15) 



\ \ ^ 1 , 

PRETEST SCORES 

Regressjon Analysis of Posttest Scores on Pretest Scores Using 
the Scores of the Students of a Sample of Teachers 

•The meu ic is arbitfary The first number in parentheses represents the rrean sco'e for the 
T s Class on the prett^t. the second numoer. the mean on the posttest 



When you do this type of analysis, you are looking for toachers who de- 
part markedly from the regression line because you want to sludy tnem 
intensively to find how they differ from teachers who are closer to the re- 
gression line The general idea is that teachers whose student performance 
shows marked departures from the line may be highly offective. other things 
being equal We ignore here the many technicalities of this type of analysis 
to make the point that the method provides a way of identifying teachers 
who may be unusual and provides a way U mductmg hypotheses about what 
constitutes performance related to student learning. 
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The next step \s to study the observed performances of these teachers 
and find the performances which discriminate their teaching performances 
from that of other teachers m the analysis If you gather enough information 
around the performance of the children and the performance of the teacher, 
such as the kinds of schools m which they are. the composition of the class, 
and similar factors, you stand a fairly good chance of sorting out those 
performances associated with the act of teaching which distinguish more 
effective teachers fror^ other teachers. 

It is not true, however^ Ih*. these performances are guaranteed to be the 
ones that are associated with effective teaching. Let us consider the prob- 
lems of this type of a study with respect to the two questions that I asked 
earlier: what constitutes evidence, and how much evidence is sufficient? 
At the end of this type of study, all you have is a set of hypotheses about 
the relationship between teaching performance and student learning that 
may have some validity. These hypotheses m st be tested either by doing 
experimental studies or by doing replications of the original study or both. 

The problem of how much evidence is sufficient arises from the com- 
plexity of these studies, it is feasible to study a limited number of student 
outcomes To gather data on students on relatively few outcomes requires 
several days of testing To study the teachers over time requires multiple 
observations. The question is» how many studies of this kind are needed 
before we have sufficient evidence of the components of competence? 
Where are they to be located in a matrix of pupil changes, types of schools, 
differences m organisational structures, differences in levels of education* 
and differences »n teacher and student characteristics before we have 
confidence that we have working hypotheses that describe competence? 

What we must not do is propose or claim that we need 500 of these 
studies and that it will take 10 years and $10 billion of research money to 
describe teaching competence. We must recognize that we are living in 
an environment of pessimism about the probability of demonstrating that 
there is a relationship between teaching competence and pupil learning. 
One of the goals of the research and development program on competency 
based training should be to demonstrate the probabiiity that teaching com- 
petence can be identified. It seems to me that several studies of the kind 
that i have described are likely to demonstrate some relationships. Such 
studies will at least eliminate some hypotheses about teaching or indicate 
weak relationships. Because such studies involve correlational designs, 
however, and because they are inductive in character, potentially valid 
hypotheses may not be discovered. Anothei type of strategy is needed that 
IS theoretical-expenmental m character. 

A iheoreticaUexpenmenlal type of study vanes teaching performances 
systematically and assesses the effects of these variations on student 
performance. The manipulation of the performances may be of two kinds. 
One manipulation is to vary the degree of the skill of a performance to 
determine the relation of the degree to the amount of the effect. Such 
experiments provide the data necessary to establish a criterion level fOr the 
performance. 

A second type of experimental manipulation requires varying the kinds 
of performances to assess their effects on the same outcomes. Such studies 

^ IS 
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provide comparativi^ d<'it<i on the effects of performances, their interchange- 
ability, and their interdependence 

These experiments should be conducted under controlled conditions 
where the conditions and materials for teaching are identical and whore 
students are assigned randomly to classes. They may also be conducted 
under field conditions using fractional factorial designs This type of destgn 
involves selecting relevant contrasting conditions m which the teaching 
occurs such as those associated with the characteristics of students* 
schools, or curricula Fractional factorial designs are notably efficient 
because relatively few teachers can be used to test a relatively large num- 
ber of hvpotheses. When used under field conditions, they provide tests of 
interactions among school and pupil factors and teaching performances. 

The advantage of experinr>ental designs is that they provide a way of 
testing tiypotheses systematically. Experiments are powerful ways of test* 
ing theories of teaching by testing the hypotheses derived from them. One 
can begin as follows select one or more models of teaching; define or 
describe the kinds of pupil changes that the theory predicts will occur If 
children were instructed by teachers applying that model; and derive hy* 
potheses about what kinds of teaching competencies produce these changes 
in students. The experiments are then designed to test these hypotheses. 

The advantage of using theories or models as starting points for deriving 
hypotheses is that the si2e of the hypothesis-pool will be greatly enlarged. 
The disadvantage >s many hypotheses will be derived of unknown validity 
either because they are freshly derived from the theory and have no empiri* 
cal support from any source or because the theories themselv<»s have weak 
empirical support At this stage of research on teaching, it is probably better 
to include more rather than less hypotheses m the pool for testing. 

A research and development strategy on teaching competence ought to 
include both mductive-correiational and theoretical-experimental studies- 
Further, each inductive-correlational study ought to be followed by expen- 
ments which test the validity of the hypotheses derived inductively. 

The problem remains, what constitutes sufficient evidence to make 
reasonably definitive statements about teaching competence? One approach 
to answering this question would be to consider the consequences of train- 
ing for a set of competencies that has been derived from an inductive* 
correlational study Here the evidence is relatively weak— -no replication 
of the original study and no experimental verification. There seems to be 
no risk in using such information, particularly if the training program studies 
what happens when its teachers use the competencies in real teaching 
situations in this case, the opportunity exists to generate more data about 
the effects of the teaching performance 

The problem of sufficient evidence seems to he m the political arena, if 
a state agency requires that programs train for the competencies derived 
from even a few inductive-correlational studies whiCh have not been repli- 
cated, one has cause for concern. 

It also seems unwise to assume that there will be an end-pomt at which 
it can be said that we now have sufficient evidence to justify" competency 
training. Rather the hypothesis-generating, hypothesis-testing life style 
should Characterise the development of the competency based movement. 
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There ought not to bf? a tutu* whiM\ w<» frwive stopped research on teaching 
competence 

A few general comments about the problem ot studying tc* mg com- 
petence are relevant here in studying teaching competence, vve have to 
separate out systems effects on children from teacher effects Many of the 
goals of education are broad Changes witf\ respect to them occur as the 
consequence of the teaching of the different kmds of teachers whom the 
child encounters and also as a consequence of other kinds of influences. 
Such as those from peers it seems to me that our basic problem in gener- 
ating a hypothesis-pool about teaching competence is to generate it in such 
a way that the systems effects are treated as potential moderators of 
teaching performances Research on competence ought not to be, at least 
initially, research on effects that occur as the consequence of Sive years 
of education Such research ts both interesting and important, but it is not 
really related to demonstrating the effects of teaching competence. When 
we know more about the specific effects of teaching acts under known 
conditiotis. we wiii be prepared to study systems effects. 

It IS important to keep in mmd that m conducting the type of inductive 
tield'Onented research that i referred to earlier, it iS possible to design 
those studies so that a wide variety of system effects may be studied as 
interacting with the effects of teaching performances. You can determine 
the kinds of influences that the administrative chmate of the school, for 
example, has on teaching performance, and how performance varies withm 
different organisational structures or withm different types of programs. 
But our problem ts to account for. or to describe how much of the variation 
in student performances actually can be accounted for by various teaching 
performances The goal of our rev .Mch is to find the effects of leaching; 
other factors m this research are important if they influence or so affect 
learning that the influence of teaching is minimal. If we cannot determine 
the effects of teaching performances, the competency based movement is 
irrelevant. 

Another problem is that we need to know how different kmds of teaching 
performances cluster - - we know practically nothing at all about the struc- 
ture of teaching performances There is some evidence that specific 
descriptors of performance cluster m factors. Most descriptions of com- 
petence are lists of behavioral acts interdependencies among them are 
made logically But if such specific acts tend to occur together in groups, 
and if these dusters have demonstrable relations to changes m students, 
then these clusters are probably the basic units of teaching performances. 

Then the question is. what ts competence'> Is it the performance em- 
braced m such factors, or is it the basic descnotors that are spelled out 
specifically and that an observer may check as present or absent in a 
teaching episode? We really do*not know, but some evidence suggests 
that perhaps it is these clusters and not the most mimscule aspects of the 
total performance that produce learn.ng effects The inductive-correlational 
type of study provides the empirical data required for the analysis needed to 
study this problem 

A fundamental problem facing us from a research point of view is: what 
IS the unit of competence'^ is it the simple semantic descriptor which is 
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useful m devising a training program or is it some combination of these 
units that produces the expected effects of a teaching performance*^ 

Permit me to summarise very briefly The focus of this presentation has 
been on the problems of research on teaching competence, which are 
problems of design;ng a strategy to identify and isolate the effects of 
leaching performances. This strategy should include two kinds of studies, 
inductive-correlational and theoretical-experimental in all such studies, 
the Criterion of effective teaching must be changes in students. Second, s 
judicious combination of designs will facilitate testing a very large number 
of hypotheses m a relatively short period of time. 

Inevitably, any conference discussing research and development pro- 
grams and problems m teaching, particularly a program directed to creating 
an empirical base for competency based trammg niust face three facts. 
First, there is a body of research whose results are mediocre and meager. 
Second, there is a body of literature and a school of thought that seems to 
Support, in the case of the literature, and has concluded, in the case of the 
school of thought that schools and teachers do not make a significant dif- 
ference in learning Third, there is widespread pessimism that research on 
teaching and specifically on competence in teaching ts ever likely to 
produce useful knowledge. 

The import of my presentation is that irrespective of what is take i as the 
present state of the art m research on teaching, methodologies may be 
systematically applied which are likely to identify the parameters of 
competence it seems pointless to continue to bemoan correlational studies 
When it IS obvious that their logical weakness m establishing causality can 
be ameliorated by replications and experimental studies. Also, the usual 
jeremiad that partialling out the influence of school and other factors is 
difficult seems pointless wher> so few researchers study teaching and its 
context. Lest I slip into a moralistic diatribe, suffice it to say that our problem 
IS to design a strategy that attacks the study of teaching competence sys* 
tematicaily and with confidence that meaningful results are likely even if 
they may not be overwhelming. 

There is one problem in designing such a strategy that will not yield to 
rational analysis alone. Tt *i research problem, as I have described it. re- 
quires first a decision about what pupil effects will be used as the criterion. 
Occasionally this choice will be predetermined as when a political agency 
stipulates pupil outcomes for which they wish to determine the antecedent 
teaching performances Otherwise the choice of outcome is made by the 
researcher. Thus, we must as a research and development community 
arrive at some consensus on the priorities of these outcomes. 

I have Chosen to place the description of competence as the critical 
problem on which research must be done. It should not be inferred that aK 
other activities need be postponed until considerable progress has been 
made on this problem. Competency based training is here both in the form 
of program development and state mandates requiring shifts to competency 
based training. Two other kinds of problems require research. 

The first of these problems is the effectiveness of training modules. That 
research is a straightforward engineering problem. The research must be 
designed to show that a particular module has produced a change in train- 
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ees Which it is e*pecteU to pfodjce One compunenl of this research is 
specifying the conditions under which you will decide that the trainees 
Should demonstrate that they have acquired a skill For example, if you us© 
microleaching you may demonstrate that und»?r the conditions of micro- 
leaching trainees have acquired a particular performance Is that sufficient 
evidence for deciding that the trainee has acquired a sKiiP Many o* us are 
accepting the criterion that at the end of the module a prespecified per- 
formance must t>e demonstrated by some assessment procedure That 
criterion is sufficient for deciding that the experimental treatment designed 
into the module does produce the desired outcome We do not know, how- 
ever, whether or not the trainee can manifest the performance under other 
conditions At some point, we must demonstrate the links between these 
specific acquisitions and the ultimate criterion of effectiveness— classroom 
performance Hence two kinds of research are needed 1 ) does the module 
produce the specified change; 2) do performances acquired at the modular 
level appear m complex performances m realistic teaching situations'^ 

Assume that a module does produce an immediate change but appears 
to nave httie influence on subsequent classroom performance. One con- 
clusion IS that the module has no transfer value to teaching But the problem 
may be withm the system of modules Many are assuming that the acquisi- 
tion of teaching skills is an additive process - that components are added 
from the Simplest to the most complex This view is useful tor an initial 
organization of a system of modules 

The modules may facilitate transfer from the acquisition of one skill to 
nnother or may inhibit such transfer Some arrangements of modules may 
be more efficient than others Some skills may be better acquired in their 
complex form rather than by accruing components of the skills. Thus, once 
one designs an instructional system, its assumptions about the composition 
of complex teaching skills should be explicated as well as the assumptions 
being made about how the skills may be best learned as the trainee pro- 
ceeds through the system. 

Each System is a model of the processes by which teaching competence 
IS acquired These models ought to be compared by treating them as experi- 
mental conditions 

Thus, a major research area is the problem of instructional effectiveness 
We need to know not only what constitutes teaching competence lo\ also 
the most effective ways to educate for it Research m ihis area should 
ultimately answer two fundamental questions what aspects of teaching 
competence are learnable and under what conditions can they t>e acquired 
most efficiently*^ 

A related problem is the evaluation of program effects. Do competency 
based programs produce better teachers? Attempts to conduct research 
on this problem at this time are premature for three reasons. The most 
obvious IS that there are not sufficient graduates of such programs to yield 
worthwhile evaluation data. Further, until programs are better developed, 
there is little point in conducting comprehensive evaluations of programs 
Second, the criteria for evaluating program effects are neither clearly 
defined nor agreed upon 

Third, if competency based programs are to be evaluated, the design of 



ERLC 



19 



BEST COPY AVAILABLE 



such t»viiiu.<t.ot\ inn}U\ to pr<,vnJf* tot comparisons of planned variations in 
the training programs As the pteceOmg analysis hopefully suggests, there 
IS ample roor>i for varying conceptions of teaching competence and for 
systematic variations .n program design We ought to anticipate program 
evaluation by planning tor such designs now These proNems must be 
solved withif^ the r^eM tew years, consequently, problems of program evalu- 
ation IS a major topic of this conference 

This discussion of research and development in competency based edu- 
cation has focused on two major problems, and by implication on a third 
the problem of mea'^urmg teaching competence Teaching competence 
cannot and is unlikely To be measured »n any meaningful way at ihe present 
time because we do not know what it is; however, research on the nature 
ot teach'.ng competence and tne effectiveness of instructional systems 
should hasten the development of the technology ot assessment. Many 
people believe that finding ways of measuring competence is urgently 
needed But. m my opinion, they seem to be seeking panaceas There are 
no tests. Observation systems, rating scales, or even simulated teaching 
s.tuaTions UMi\ can be used m any sensible way except m the context of 
research on competence Thus, a major concern of this conference is the 
use of assessment procedures m research on competence and instructional 
and program ett«>ctiveness 

This contorence will also give attention to two other problems that require 
study The first of these is related to the social context of the schools. 
Teachir\g skill is influenced by the composition of the staff and students of 
a scfiooi. by the orgamration of the staff and students of a school: by the 
social miheu ot the schools, by the value consensus of staff, students, and 
community and by curriculum designs Thus, the study ot competence is 
necessarily incomplete until it is studied in the context ot schools, traditional 
or alternative, open or structured 

Teaching competence has multiple social meanings, and these meanings 
define the parameters of researct\ on teaching effectiveness Both the en. 
ter.a and conditions of effectiveness vary as the social context of education 



vanes 



ERIC 



The last problem that this conference will consider is the public policy, 
or more mundanely, the political implications of competency based educa- 
tion There are two categories of problems to be considered in this respect 
First are the problems ot change withm training institutions Competency 
based programs require changes m roles and habits, the structure of inter- 
personal relations among faculty and students, and the assumption ot public 
accouruabihty not heretofore either expected of or required by t?.e trauung 
institution We need to know the factors that facilitate or t iterfere with such 
change-., the problems encountered m change, and the most effecMve ways 
of resolving them 

Second are the prc»* -ms ot public policy These problems relate to the 
control of tf\e framing of and certification of teachers and the criteria for 
and evaluation of leachcr effectiveness The mandating of certification on 
the basis of compeie.nce is a matter of pubhc policy with significant impli- 
cations tor institutions and organizations whose interests are affected by 
such major shifts m public policy We know very httle of a systematic char- 
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acter about these political ptohlimis. though son*i5 of us may have practical 
experience with them it is difficult at this time to visualise what research 
needs to be done. t5ut the competency based movement is immersed m the 
pohttcal process What issues of this character need to be studied and how 
should thcy be studied'' 

In conciusiorv this conference will address these five problems 1) what 
is teaching competence. 2) what are the most effective ways of educating 
for It. 3) are competency based programs producing effective teachers. 
4) What are the influences of the social context on teaching competence: 
and 5) what are the public policy issues relevant to competency based edu* 
cation*? 

The purpose of the conference is not to provide answers to these ques- 
tions because there are no simple answers to them Its purpose is to clarify 
the problems, to propose appropriate research strategies for studying thenr*. 
to bring Current experience mto the analysis of the problems, and to stimu- 
late interest m a research and development program that wUI Jndergird 
the competency based movement 

Today is hopefully the beginning of the creation of a national research 
and development oftort whose ultimate purpose is through the study of 
teaching and us effects to influence the ^^uahty of education and to provide 
the knowledge necessary to make the most profound and far-reaching 
processes for cr^ange in teaching successful 
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2 National Institute of 
Education and 
Research in C/PBE 



Garry L McDanMt 

The purpose of this conference is to focus on the question What are the 
things that teachers do which make a diflerence for children*?" This ques- 
tion 1$ reasonable Our busmess »s to help children, however, as unreason- 
able OS It may seem, the quantity of research linking teacher training and 
teachi^r performance with child performance is limited; some would say. 
nonexistent The National institute of Education (NiE) plans to ask this 
question repeatedly over the next several years in its Program on Teaching 
and Curriculum in asking this question, we hope to examine many of the 
assumptions underlying competency based teacher education and licensing 

In 4he pages that follow, the process by which the NIE will plan its re- 
search efforts in this area will be described The first section argues that 
thv% current relationship between funding agencies and researchers could 
be made more efficient. The second section describes how to establish a 
more efficient relationship between researchers and funding sources The 
third section describes one phase of NiE s planning efforts— the National 
Conference on the Research on Teaching 

While these efforts are more broadly focused than the issues related to 
competency based licensing and certification, parts of this effort will 
obviously be relevant to the primary concerns of this National Conference 
on the Assessment of Competency Based Teacher Education and Licensing. 

Jh% Rational* for Dtvolopmtnt of a Rotoarch Plan 

The purpose of this section is to describe the efforts Of the National 
Institute of Education to create a relationship between itself as a funding 
agent and the research and development community. While the focus is on 
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NiE and Its relationship, the principles apply to the more general issue of 
the relationship between any funding agent and its constituency 

The assumption underlying this discussion is that relationships between 
funding sources and researchers, especially m the social sciences, are 
ready to imcomo more efficient, that these more efficient relationships will 
improve the impact of social science research on contemporary problems, 
and that improved relationships will also enhance the :^age of the social 
sciences as an effective tool for attacking social problems M is ME s 
premise in this planning conference that a major path toward improving 
efficiency Will result from improving the ability of researchers and funding 
sources to develop research plans 

Iff Currtnt Rtlattonthtp Bt twon Most Funding AQtnetat 
and Tbtir Rtlattd Rtttareh CommunHlaf 

At present, the process for funding education research is as follows. The 
researcher develops proposals m isolation from the funding source Pro* 
posals are sent to several funding sources where some rather abstract 
objectives seem consistent with the mtent of the investigator if a source 
finds something interesting m the proposal, a peer panel is usually assigned 
(a) to assess its internal Quality, and (b) to consider the merits of the pro- 
posal as It relates to some field ' Finally, a set of the most highly rated 
proiects a»e matched against a budget, a budget usually determined prior 
to the submission of ideas. 

Several sources of inefficiency can be seen here the first is the problem 
of not knowing where to send a proposal The second iS that, even if the 
appropriate funding source is found, it is necessary not only that the general 
content of a proposal be adequate, but that the proposal emphasis be related 
to the primary concerns of the funding source. Often, the proposal is wide 
of the mark at the time of the evaluation. The third source of inefficiency is 
that a collection of good proposals that add up to a program strategy must 
often be arbitrarily cut because a budgeting level was set prior to the 
establishment of strategies for attaining the goals of the funding source. 

The efficiency and effectiveness can be improved if four essential types 
of activities occur 

• The abstract goals and general priorities of the funding agent should 
be established 

• Specific ob}ectives and strategies should be developed 

• Individual projects being funded should meet criteria of internal 
Quality as well as criteria which require that one projec; relate to 
others in a cumulative fashion. 

• All projects, strategies, and objectives should be disseminated m 
public documents so that public recommendations and budget alio* 
cations come before the actual fundmg 

It will b€ productive to consider the parameters of these activities 
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Estabiishing God/5 ^ind PftDtift^s 

Mow caretuii> are the oDiettivfs of .1 tiirOmg .igency st^ited'' let us 
consider tor example the legislative misdate ot an organizalion such as 
the National institute ot Education it is hroadiv stated The authori/mg 
legisl.it'on .isKsthe N'E \o 

• Help solve Of alleviate the prat)!en\s ot ar\(i .u hiOve the Ojectives ot. 
Amer'can education 

• Advance the practice ot education as an art science and protession 

• Strengthen the scientitjc and technological toundations ot education 

• Build an etfective educat'on research and development sys\em 
Many education res*»arrh sources especially private sources, do not 

otten detine goals bevond the abstract level of a mandate The institute has 
recogni/ed this problem Theretore. one ot the initial maior actions ot the 
National Council on Education Research jNiEs policy-making body! was 
to increase the specificity of NiEs mandate by identifyir\g priority areas 
The priorities tocus the worK of the Institute in five areas 

• lsser)tiai Skills investigating ways to aid all children m obtaining 
skills essential to function m sooety initial emphasis will be placed 
on reading i$1 . miilionj 

• Productivity ensur.ng the Nations continued ability to meet its 
commitment to quality education at a price it can afford, through 
improvemenis »n the product'v.ity and efficiency of the education 
systerr^ i$16 5 million) 

• Education and Work improving our understanding ot the relationship 
between education and work, and improving career access and pro- 
gress«on <S21 3 million) 

• Problem Solving accumulating the knowledge and experience 
necessary to assist state education agencies, communities, and 
their local schools to build the capacity to address local problems 
and adapt solutions suited to local needs ($8 6 miihon) 

• Diversity improving our understanding of individual and group 
differences in learning styles, needs, and preferences and developing 
programs that take those differences mto account ($20 3 million) 

While these priorities reflect progress toward specificity, the statements 
descr't)e only the general ci-.aracter of NiE s concerns, they do not articulate 
the essential character of the problems being attacked by the funding 
source 

Developing StrategieB 

Funding agencies seldom articulate strategies tor achievmg their objec- 
tives or provide a rationale tor the strategy adopted it is exceptional to 
find u strategy clearly art'Cuiated m advance ot making funding decisions 
This point can be illustrated m some of the majof curriculum development 
efforts These efforts have led to the question ot what to do next There is 
some skepttcism about the extent to which the national curriculum efforts 
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such as new math provMltMj ( iniijrpn with new mathematic understanding 
and skills 

Assuming that the skepticism is reasonable, researchers have three 
general strategies regarding what to do nent One strategy is to do nothing 
if the outcomes are uncertain let us no ior>ger support large curriculum 
efforts, let us focus our efforts, for enample. on helping teachers to develop 
their own curriculum A second general strategy is based on the argument 
that such projects may not have really been well implemented m class- 
rooms Teacher training may have been inadequate, or materials may not 
have been sufficiently prepared The »ssue becomes the process of imple- 
menlauon A third strategy »s based on the arguments that we really do not 
have measurement devices which are sensitive to such a sophisticated 
Curriculum vanatioti This problem is essentially one of measurement This 
third strategy argvies that measurement issues must be given high priority 

While this IS not an elegant illustration, it indicates how a public airing 
of one or more strategies would be helpful m getting the field of researchers 
to debate a common issue And when a strategy is selected, knowledge of 
that strategy would improve the efficiency of the relationship between 
researchers and the fund^g source 

Evatuattng and interrelating Research Proposals 

The third activity that should occur m devolopmg plans is the develop- 
ment of strategies for evaluating research projects and proposals as to 
lhe»r (a) internal quality or logic, and (bi the extent to which one project 
relates fruitfully to other projects 

The first criterion has had prtmacy The quality of the internal logic, 
design, analytic strategy, etc provides ihe essential basics tor project 
evaluation But there should be a second criterion 

The second criterion receives notice but rarely any systematic analysis 
Any evaluation of the degree to which projects are cumulative is usually 
left to those who review the literature, and such reviews are properly 
suspect It IS even difficult to organize symposia that put together projects 
that are additive or that produce a substantial mass of information related 
to a topic Without a cross-project analysis, pnor to funding, a cumulative 
impact IS left io chance There is the continuing assumption that if a project 
IS intelligently designed and is funded, that project wHi somehow comple- 
ment the next project funded Even more optimistically, it is often assiurned 
that, when all the projects are completed, the magnitude of some major 
problem will be reduced 

Communfcatmg Profects, Strategies, and Objecttyes 

The fourth and last activity needed is communication to make certain 
that a research plan exists which can define the relationship between fund- 
ing agencies and their research communities. Sufficient and timely informa- 
tion regarding the objectives, strategies, and evaluation procedures of a 
research agenda is seldom available There have been few. if any. instances 
m social science research where a public debate about the essential 
elements of a research agenda has occurred prior to funding. Although a 
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catalog of funded ptoiects is usually available, it is rarely possible to obtain 
a catalog of proposed projects 

Establishment of an Alternative Relationship 

It IS possible to create a relationship between funding agencies and their 
related research communities based on the foregoing four kinds of initia- 
tives An apDroPCh is being tested m the Program on Teaching and Curricu- 
lum (directed by Garry L McDamels) and m the Essential Skills Program 
(directed by Marshall Smith) at the NatiOT>al Institute of Education. After 
the general approach is described, the specific initiatives being taken will 
be presented. The general approach has four elements; (a) specifying 
major problems, (b) obtaining simultaneous input from field researchers 
regarding possible projects, (c) establishing alternative strategies for 
attaining goals, and (d) communicating alternative strategies with project 
recommendations. 

SpBCifymg Major Probiems 

Instead of having the funding agency identify major problems at a broad 
or abstract level, the agency must force concrete definition of problems. A 
Federal funding agency cannot remain at the general level of the goals of 
the authorizing legislation, just as the private 'oundation cannot effectively 
use the broad goals identified m its request to the internal Revenue Service 
fur tax-exempt status. Also, if the character of the problems are not fully 
descnbeJ. it is also rather difficult to assess the accomplishments of the 
effort 

Obtaining Simultaneous Input from Field Researchers 

Investigators are storehouses of project ideas. As time and conditions 
allow, an idea is selected, refined, and sent to a funding source by an 
investigator Three problems result: 

• The ideas are sent from different people at different times 

• It is difficult to relate projects across agencies and foundations 

• Each person's proposal is developed in a time lag from the recent 
work of other investigators; therefore, he may plow the same ground. 
If cross-project analysis is to be possible, it is essential that simul- 
taneous input from field researchers be achieved. 

Establishing Alternati\^e Strategies 

Not only must the input from field researchers be obtained simultaneously, 
it must also be obtained m a manner which identifies the intellectual and 
administrative relationships among projects. The development of a plan 
requires that the potential research ideas be organized into a logical agenda. 
Further, the organization should demonstrate how the projects relate to 
the achievement of an objective worthy of public or other support. 

Communicating with the Research and Development Public 
Finally, the steps described above provide information to be communi- 
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caied to the research and development public Much of that public will be 
informed as a result of having been a part of the process. The remainder will 
have the time required for receiving and analyzing the pertinent documents. 

The National Conference on Studies In Teaching 

To further clarify the rmplications of these elements, the steps being 
taken m the Program on Teaching and Curnculunn will be described. In 
regard to specification of major problems, the staff is meeting with stu- 
dents* teachers, administrators, teachers m colleges of education, etc. 
to get a summary of problem statements. These problem statements will 
serve as a charge to the researchers who will make project recommenda- 
tions during the planning conference. Perhaps it would be ideal to have 
everyone meet simultaneously. Such a meeting would, however, require a 
quota system for representation of the relevant groups— which would 
result in huge working sessions that would be impossible to manage. 

To obtain simultaneous input from field researchers, the present con- 
ference has been planned. To establish the organizational framework for 
the development of the plan, the panel chairmen will spend several weeks 
m making a separate analysis of subject area objectives, in cooperation 
With the conference participants. During the conference, the participants 
Will refine those approaches, identify possible areas of research to imple- 
ment the approaches, and specify the research projects which should be 
considered by the Agency. 

Finally, to communicate thesa proceedings, several steps will be taken. 
First, all participants will leave the conference with complete volumes of 
panel suggestions. These will be edited for minor errors and sent to the 
Government Printing Office ^ur printing and distribution. The analyses by the 
NiE which result in strategy recommendations will be available for general 
distribution at the end of August 1974. The logistics of printing and dis- 
tribution have been established. 

Substantive Form of the Panels 

Preliminary discussion by NIE staff and consultants smce August 1973 
has fielded a formulation of the substantive structure of the conference— 
and the consequent foci of its 10 panels. In part, this structure is based on 
a categorization of major concerns m the career of the teacher: (a) recruit- 
ment, selection, and retention: (b) training, performance, and effectiveness; 
and <c) utilization In addition, three foremost concerns of all Research and 
Development m teaching will be taken into account by panels on Personnel 
Roles in New Instructional Systems. Research Methodology, and Theory 
Development Thus, the Conference will consist of the following panels: 
1 Panel on Teacher Recruitment. Selection, and Retention 

2-6. Panels on Teacher Training. Performance, and Effectiveness 
2 Panel on Teaching as Human Interaction 

3. Panel on Behavior Analysis m Education 

4. Panel on Teaching as Skill Performance 
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5 Pc4ru»i im Itsu:hiru} as a Logical and I inguistic Process 

6 Panel on Teacrnny as Clinical Information Processing 

7 Panel on Teacher Utilization 

8 Panel on Personnel Roles m New Educational Systems 

9 Panel on Research Methodology 

in the tollowing paragraphs, each of these panels js briefly character .zed. 

Panel i Teacher Recruitment. Seiection and Retention 

The panel on Teacher Recruitment. Selection and Retention wHI be con- 
cerned with planning Research and Development on deviS'ng. validating, 
and installing methods for improving formal and informal decisions at 
various points in a teacher s career 

(a) admission to a teacher education program 

(b) retention in the program 

(c) admission to student teaching or internship 

(d) graduation from the program 

(e) certification or licensure 

(f » employment by a school system 

(g) granting tenure 

(h) promotion in rank and pay 

The Classical paradigm m Research and Development on recruitment, 
selection, and retention consists of identifying criteria, selecting potential 
predictors of those criteria, and determining empirically their predictive 
validity. The panel will accordingly pay much attention to the matters of 
criteria, predictors, and ways of basing decisions on their relationships. 
The criterion problem will entail consideration of competency assessment 
and teacher evaluation procedures. 

The panel will also deal with self-selection—ways in which persons are 
attracted toward or turned away from teaching, i.e . the occupational choice 
and attrition processes. It will examine the social class, ethnic, and other 
origins of teachers and the differences these make. It will also deal with 
the possibility that success m a training program is more a function of the 
program than of the candidate and his characteristics 

Panel 2: Teaching as Human Interaction 

Many studies have dealt wi*h observed human interaction (either live or 
recorded) as independent and dependent variables. In some cases, obser- 
vations of human interaction have been made merely to describe differences 
between kinds of teachers m various settings and contexts. Tiose studies 
have been reviewed m several monographs. This approach has been the 
most frequently used m research on teaching during the past two decades. 

To review the current state of this approach, to lay out the most promising 
next steps (such as projects on taxonomies of teacher behavior, chains and 
patterns of teacher and student moves, computerized storage and analysis, 
and archival banks of videotaped interaction records), and to formulate 
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those projects in optimally definite terms— ^these illustrate the tasks that 
can be undertaken by a panel m this highly cultivated, but far from ex* 
hausted. field. 

* Research and development on teacher trammQ. performance, and effectiveness deals 
with teacher variables (toacher characteristics and behaviors) m either or both of two ways. 

(a) as dependent variables (with teacher training methods serving as independent variables), 
and 

(b) as independont variables (with effects on students serving as dependent variables) 

Work m this ar<»a has been based on a number of paradigms, each relatively independent. 
Hance. several corresponding panels (2-6) will work here 

Training is construed to etn brace socanzat on processes mat imbue the trainee with the 
values and norms of the profession it includes both preservice and mservice training These 
panels will emphasize aspects of training that require practice, i.e . training intended to 
provide 'knowledge how." not merely "knowledge that " The panels will be encouraged to 
develop new paradigms, cutting across or departing radically from present ones 

Panel 3 Behavior Analysis in Education 

Somewhat more recently, but with equal vigor and productivity, the 
behavior analysis in education approach has been applied to the improve* 
ment of teacher training, performance, and effectiveness. 

The success of teacher training and especially of classroom management 
by these methods has been frequently reported. Thus, this thriving line of 
attack on several important problems in teaching and teacher training 
merits the attention of a panel. This panel might seek to increase the atten* 
tion already being given to private events and cognitive processes. It could 
advance the effort to apply behavior analysis in education to the achieve- 
ment of humanistic educational objectives, it could consider studying the 
effect of students on teachers through the cues provided by students, 
perceived by teachers, and responded to by teachers, and thus the ways in 
which students may reinfoice teachers* responses. 

Questions have been raised about the morality of "controlling** behavior 
through behavior modification methods. Hence, this panel may undertake to 
address such ethical issues. 

Panel 4: Teaching as Skill Performance 

During the past decade, an analytic approach to the training and evalua- 
tion of teachers has frequently been adopted. In such an approach, teaching 
is broken down into component skills, such as questioning, explaining, 
listening, and reinforcing. Little concensus has been reached on what skills 
should be identified But the approach itself has flourished, lying at the basis 
of such innovations as microteaching and minicourses. in which teachers 
are trained in only one or a few skills at a time. 

The analytic approach has also led to the movement to develop teacher- 
training products. Such products call upon the trainee to practice a given 
skill, in addition to reading, listening, or watching material about the skill. 
Hundreds of such pro tucts have been developed and cataloged, in part as 
a response to the nationwide attempt to make teacher training and evalua- 
tion "performance based.** Their promise will be realizeo. however, only to 
the degree that they benefit from further research and development. The 
analyses should become more valid, logically and psychologically. 
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Panel S: Teaching as a Logicaf and Linguistic Process 

Studies of linguistic and logical aspects of classroom interaction here 
and in England have varied widely in the particular aspects of language 
studied, the kinds of classrooms selected for research, and the methods 
used for data collection and analysis. 

Aspects of language studied have included the logical operations 
engaged in by teachers and pupils: the types of pedagogical 'moves** in 
the class. com interaction 'game" and their distribution among teachers 
and pupils: formal speech registers used in the school, as contrasted with 
the home, and their various special characteristics in vaiious school sub* 
jects: and the sociolinguistic rules operating in the classroom seen as a 
special setting tor face-to-face interaction. 

Most research has been conducted in teacher-structured recitations, 
less in small-group and pupil-led discussions and group projects. Many 
researchers have assumed a set of norms governing classroom interaction 
that is Shared by ail participants: others have deliberately worked in multi- 
ethnic settings and considered variations in these norms as a special 
problem for analysis. 

On the basis of this previous research activity, this panel will undertake 
to formulate the direction in which research should move to provide im- 
proved understanding of logical and linguistic phenomena in classroom 
settings and the ways in which that understanding might be used to improve 
the teacher *s work 

Panel 6: Teaching as Clinical Information Processing 

This panel should be concerned with the thought processes that precede 
the teacher's choices of action. How teachers think, make judgments, size 
up situations, plan and react to unanticipated contingencies — all these 
should be of concern here. Thus, this panel would be concerned with the 
study of teaching as thinking, problem solving, judgment, and decision 
making, it might also consider the value of a clinical (possibly anthropologi- 
cal) approach to research on teaching. Studies of the teacher as clinician 
would be concerned with the way in which theory, although it does not pre- 
scribe for practice, nonetheless informs practice. 

The diagnostic approach to teaching and teacher preparation would also 
be concerned with the teacher's expectations. Such expectations have 
been found to influence not the student's intelligence, as was once claimed, 
but teacher behavior, teacher-student interaction, student motivation, and 
student achievement of the objectives at which instruction is aimed. This 
panel should, therefore, gii/e attention to the determiners and effects of 
teachers* oxpectations concerning students in various categories (sexes, 
races, ethnic groups, age levels, etc.). 

Panel 7: Teacher Utilization 

After teachers have been selected and trained, they go to work in a school 
Which is part of a school system. What happens to them in the months and 
years of their work in the school obviously influences greatly the ways in 
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Which they teach, tee! and. what is most important, influence their students. 

The term utilization refers to the ways tn which teachers are required or 
permitted to work by the administrative, organizational, and physical en- 
vironment that their school and school system provide. The many new ways 
of *utifi2;ng ■ teachers in this sense include team*teaching. teacher aids, 
nongraded classrooms, schools without walls between classes, accounta- 
bility laws, mandated teacher evaluation systems, voucher plans, teacher 
centers, flexible scheduling, and differentiated staffing. How is academic 
labor divided? What are the causes and effects of what is often considered 
to be a low levei of collegtality m elementary and secondary schools? Should 
utilization variables be used m studies of school outcomes so that their value 
can be assessed'^ Can relationships between schools be arranged so as to 
improve teaching and mservice teacher education? 

Sociologists, school administrators and teachers' organizations have been 
especially active m developing and analyzing innovations in teacher utiliza- 
tion The panel here should contain representatives of these groups. 

Panel 8 Pt>rsonnel Rotes in New Instructional Systems 

Some writers regard the traditional classroom organization of the school 
and the teacher's role in that clas<;room as moribund. During the next two or 
three decades, the presently prevalent arrangements will be supplanted by 
what are now regarded as revolutionary new arrangements. The new schools 
Wilt incorporate instructional systems that fully exploit the potentialities of 
computers, television, and various other products of recent technology. 

The role of the teacher will change in correspondingly radical ways. In 
some views, the teacher will have much less to do with the intricacies of 
facilitating achievement of cognitive objectives, because computer-assiated 
instruction will carry much more of that burden. Instruction will be more 
effectively adapted to the needs and capabilities of each individual student. 

Such individualization will adjust the pace, the occasions, the Initial 
phases, the remedial efforts, and the media or methods of instruction so as 
to optimize learning of the individual student. Beyond these kinds of adjust- 
ment, the new systems will adapt the subject-matter presentations to the 
individual's aptitudes, learning styles, attitudes, preterred types of rein- 
forcement, and immediate past history of responses. They will also use new 
approaches to the evaluation and grading of student achievement, turning 
more to "criterion-referenced" approaches. In turn, these approaches will 
lead to the objective of greatly increasing, indeed maximizing, the per- 
centage of students that master all aspects of all their courses. Such 
mastery " ideals will be achieved much more abundantly in the new instruc- 
tional systems. 

What Will be the teacher's role m the new systems of instruction? Will 
the teacher become less the curriculum planner, the diagnostician and 
orescriber. the evaluator'> Will the teacher become more group process 
facilitator, the mediator of emotional responses, the model for moral and 
esthetic development^^ 

This panel will develop approaches to the research and development that 
will provide bases for predicting and optimizing the roles of various kinds of 
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persons m the new msttuctional systems Basing its work on a variety of 
conceptions of the new systems, the panel will explore and formulate pro* 
posdis for ways of making the emergence of the roles as efficient and 
rational as possible 

Panel 9. Rosoarch Methodology 

As in most fields of the behaveoral sciences, methodology m Research 
and Development on teaching, of the kinds with which panels 1-8 are con* 
cerned. has suffered from inadequate standards of validity. 

The primary purpose of the panel on methodology is to identify method- 
ological problems that appear crucial to the successful completion of 
research on teaching. The pane) should produce a series of brief descrip- 
tions of problems that can be integrated into one component of a directed 
research program The results of the directed research program component 
are to be methodological developments that are applied in the sense that 
they are of clear utility for research on teaching. It is also hoped that the 
directed research efforts will contribute to the more general field of psycho* 
metrics 

There are a number of methodological problems which need further 
attention and that relate directly to the value of research on teaching. In 
the area of measurement, there is a need for better understanding of the 
properties of existing measures of independent and dependent variables 
commonly used m past research. Such difficult problems as measuring the 
Quality of a teacher (in several respects and across several situations) 
require new instrument construction. There is also a need for improved 
understanding of existing design and analysis strategies that would increase 
the utility of future research. 

The Panel on Methodology will concern itself with both types of problems, 
with emphasis on the latter, in addition, it may call for a sampling survey of 
the methodological problems and errors in the recent literature on Research 
and Development on teaching. The finding of the survey might lead the way 
to preparing a set of methodological standards and guidelines for research 
and development in teaching. If they proved effective, such standards and 
guidelines would raise substantially the validity of the work of the next 
decade. They would also enhance materially the degree to which research 
findings would become comparable across studies, so that future reviews 
of the literature would be able to accumulate results in valid ways. Of 
course, at the same time it is necessary to remain sensitive to the ;deal that 
methodology should be especially well suited to the particular investigation. 
Therefore, it would also be useful to develop methods for synthesizing 
results from studies having different methodologies. 

Summary 

This. then, is the strategy being tested by the NlE to increase the impact 
of the research on teaching. If the strategy is only marginally successful, 
we Should move a long way toward providing the data needed to answer 
the question: "What are the things that teachers do which make a difference 
for children 
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3 Closing the 
Knowledge Gap* 



H. Del Sehalock 

The position paper prepared by the Consortium of CBE Centers has three 
purposes The t>rst is to suggest the kinds of research needed to C>ot>e the 
knowledge gap that now exists with respect to competency bdied teacher 
education The second is to build a case for CBTE programs as unusually 
rich contexts for basic research in education The third is to ouiline a frame- 
work for proceeding with basic research that will yield information of greater 
use to practitioners than have most basic research efforts in education m 
the past 

Background 

the paper stems from the experience of the elementary model directors 
m attempting to design and implement CBTE programs m their own insti- 
tutions, or in attempting to help others do so In the course of these efforts, 
ail have cxpeiienced the frustration of not having sufficient information on 
which to base program design decisions, all have recognized that there is 
little hope of improving the quality of what ts being done until there t$ in- 
creased knowledge of what works and what does not work: and ail have felt 
concern about the rapid move to adopt CBTE programs without dear evi- 
dence that what IS being adopted is any better than what has been in opera- 



* This IS H su^rnary ol a position paper prepared by the research corrimittee of the Con* 
sortium ot CBE Centers on CBTE programs as a tocus ol and context for research m education 
Memftets of the research committee are Or vere Devauit university of Wisconsin. Or 
Norman Dodi Fior«da State U-»ivefSity Dr Bruce Joyce Columbia University, and the autrior 
The paper -s bemq puDi«s»^ed «n full by tne Mu't- State Consortium on Performance Based 
Teathe* tducat«o»^ A comprehensive bini.oqraphy is included m thai monotjraph 
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tion in the past The program of research that has been proposed by the 
Consortium of CBE Centers (s designed to overcome these shortcomings, 
and in the cou«^se of so doing to move both education and teacher education 
closer to a mode of operation that is empirically based. 

!n addition to the pressing need to know, three other conditions have 
contributed to the decision to prepare a position paper on research m CBTE 
at this point m time. The first is that CBTE programs are reaching a level of 
maturity that permits the research called tor to be earned out. Research on 
competency based teacher education, for example, or research that com- 
pares CBTE to non-CBTE programs, reauires operational CBTE programs. 
Now that such programs exist, the essential condition for such research has 
been met 

The second condition that triggered the preparation of the position paper 
by the Consortium is the increasing demand for evidence as to cost and 
benefits associated with CBTE Persons who control funds m support of 
teacher education programs and persons who have been led to expect 
significant changes m the educational system as a function of CBTE pro* 
grams, are demanding to see data m this regard Providing such data is 
seen as a fundamental obligation of the Consortium, as well as a necessary 
condition for the survival of the CBTE movement generally. 

A third factor that has contributed to the preparation of the paper is the 
inconsistency that presently exists between CBTE model expectations and 
CBTE program implementation so far as a data based mode of operation is 
concerned Ail of the elementary models called for continuous program 
adaptation and for this adaptation to be based upon empirically derived cost 
and effectiveness data With tew exceptions this basic tenet of the models 
has not been realized, and to maintain any degree of credibility— as well as 
to optimize the power of model based programs— tt must 

Overview 

The paper outlines four diff vent kinds of studies that need to be under* 
taken if the knowledge gap with respect to CBTE is to be closed for all 
concerned. These have been labeled Policy Oriented Studies." "Practice 
Oriented Studies. "Basic Research Studies." and "Documentation Stu- 
dies ' 

As used tn the paper. Policy Oriented Studies ' are essentially cost- 
benefit studies of CBTE versus non-CBTE programs, or of CBTE programs 
that differ m major respects, whereas Practice Oriented Studies" are 
essentially cost-effectiveness studies of alternative approaches to the 
mastery of teaching competencies within the context of CBTE programs. 
The knowledge coming from practice oriented studies, of course, is as 
applicable to non-CBTE programs as it is to programs that reflect CBTE 
characteristics 

By contrast. Basic Research Studies" are conceived as dealing funda* 
mentally with two matters The first is the search for new or more powerful 
constructs that can be applied to the field of education or teacher education. 
The second is the application of these constructs m the search for principles 
of tnstructio'' As m the case of practice oriented studies, knowledge coming 
from basic research studies is equally applicable to CBTE and non-CBTE 
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The proposal to carry out Documentation Studies is m response to the 
need tor usable case histories on the development and implementation of 
CBTE programs Because of the relative complexity of CBTE programs and 
because of the general lack of familiarity with many aspects of their opera^ 
tion help needs to be g»ven to program aesigners and developers at a nuts 
and bolts level Documentary studies are seen as providing this Kmd of 
help Three kinds of documentary studies are called for in the paper one 
that describes the development of programs one that describes the struc* 
ture and operation of programs, and one that describes the diffusion of 
pr ograms 

Because of space limitations, the research program tsmg called for by 
the Consortium of CBE Centers is treated m the present paper essentially m 
outline form This will be bothersome to some, for the level of detail needed 
to see What the research might took like when earned out is missing. To 
others, however, this lack of detail may make it easier to place m perspec* 
tive the overill program of research that is being proposed The reader who 
wishes greater detail than is offered m the present paper is referred to the 
Muiti- State Consortium publication of the position paper m full (Schalock. 
19751 



Part I: Policy Oriented Studies 

Persons responsible for the allocation of resources m support of teacher 
education need to know whether the educational returns from CBTE pro- 
grams are greater than those from traditional programs, whether the costs 
associated with CBTE programs are greater than those associated with 
traditional programs, and the relation of costs to benefit*; m both cases, 
information of this kind is needed by persons responsible for establishing 
state and national standards for teacher preparation programs, by insti- 
tutional executives who must allocate resources m support of programs, 
and by employees of the Federal government who must recommend or 
make policies relative to the funding of Federally sponsored activities that 
relate to such programs Without such information, policy decisions must of 
necessity be made on some basis other than fact 

Two lines of research need to be undertaken m order to obtain information 
that facilitates policy decisions The first involves the comparison of CBTE 
and non*CBTE programs as to Costs and benefits The second involves the 
same kind of comparisons, but of CBTE programs that define teaching 
competencies at different levels of generality, or at the same level of 
generality, but m markedly different ways For ease of reference these two 
designs are referred to as 

• a CBTE/non-CBTE design, and 

• a CBTE Program Variations design 

Three conditions are essential to the implementation of either design. 
First, the criteria that define and differentiate desired program variations 
must t3e clearly specified Second, ongoing teacher education programs 
must exist that meet these specifications. Third, persons responsible for 
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such programs must be v^tlhny to take part m a comparative research study 
Put into other terms, the CBtt/non CBTE research design caHs tor clarity 
as to what constitutes significant program differences, essentially pure 
cases" of mature, fully functional programs that reflect these differences, 
and teacher educators willing to engage m a comparative research effort. 

Given these conditions, policy oriented studies in the arena of CBTE will 
probaoiy take the form of natural' field experiments- that is. systematic 
comparisons of planned variations m already established programs. 

CBTE/Non-CBTE Studitf 

The most critically needed policy onented studies involve the comparison 
of CBTE and non CBTE programs Because there are major differences 
within CBTE and non-CBTE programs, however, and because these differ- 
ences may be as significant m terms of cost/benefit relationships as the 
differences between CBTE and non-CBTE programs, an effort should be 
made to include within such studies a sampling of the major variations 
that appear withm both CBTE and non-CBTE programs On the CBTE side 
this should involve programs that define competencies primarily in terms 
of knowledge, ^kiii. and the ability to perform the jo*^ of a teaclioi in a 
school setting ' On the non-CBTE side, programs should be selected that 
reflect the classic, discipline centered, four year undergraduate approach 
to teacher educaiiun. phenomenotogical" approach to teacher educa- 
tion, and perhaps the Master of Arts m Teaching (MAT), or the Teacher 
Co^ps aoproach to teacher education These possible variations m program 
design are shown schematically m Figure 3 1 

Figure 3.1 

PROGRAM VARIATIONS THAT NEED TO BE TAKEN 
INTO ACCOUNT WHEN COMPARING CBTE 
AND NON-CBTE PROGRAMS 



Dominant Variations m 
CBTE Programs 



Dominant Variations in 
Non-CBTE Programs 



1 Knowledge Centered Definition 1 Classic Four-Year Under* 



ot Competency 

Skill Centered Definition 
of Competency 

Job Performance Centered 
Definition of Competency 



graduate Approach 

2 Phenomenological Four-Year 
Undergraduate Approach 

3 Mat or Teacher Corps Approach 



In Carrying out Studies ol this nature, it would seem that ui least three 
programs would have to be found that meet the requirements of a particular 
program venation m order for that variation to t>e included in a study Oper* 
ationally. this probably means that the programs studied would center at 



• According to Tufnof s cr-tettd classifying Ihe loCuS ol CUTE programs iTurnCf W^). 
knowieOge ceniorM ptogram*. focuf* ai level "ifciM'Conterert prQgr«im& locus at ?..e. 
lojr or tr^re^. and )0t> performance rf^nicired program^ locu5 ai tovDis two or one 
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the eiementaty (ev<?(. to? it ts floubtfut thrit a sufficient number of fuily de- 
veloped C6TE programs ex«st iit the secondary level to permit the design 
requirements of three programs per ceH to be met 

In addition to controlling as carefully as possible for program differences 
at the treatment level, the CBTE/non CBTE research design requires that 
careful attention be given to the output or dependent variables to be con- 
sidered, the unw.inted sources of variation to be controlled, and descriptors 
of program cost fable 3 1 contains a summary of pielimmary thinking about 
such matters 

Program Variotion Sludlea 

This design is simply an elaboration of tr^ft CBTE portion of the design 
described previously m the CBTE/non-CBTE design, it was proposed that 
CBTE programs included m a study vary by level of competency definition. 
In the present design it is proposed that within level differences be explored 
as well. 

Operationallv. the CBTE Program Variations design calls for the identifi* 
cation of programs that define competency primarily m terms of knowledge, 
skttl or )0b performance, and for the comparison of major program varia- 
tions withm each of these three levels of competency definition as to the 
effectiveness of their graduates on the same criterion measures as em- 
ployed m CBTE/non-CBTE studies All of the conditions that need to be met 
in order tc carry out CBTE/non-CBTE studies must be met m carrying out 
CBTE Program Variations studies, for example, mature CBTE programs of 
varying foci and emphases must be available, people m those programs 
must be willing to engage m a comparative study, and the same unwanted 
<:ources of variation must be controlled The same dimensions of program 
cost must also be attended to 

Unfortunately, given the relatively limited development of CBTE programs 
nationally, it is likely that this design wil! be applicable in the near future 
only to programs focusing at the knowledge and skill levels. There simply 
are not enough CBTE programs m operation at the present time that define 
competencies at the job performance level to permit the comparison of the 
effects of alternative program emphases to be meaningful 



Part II: Practiee Oriented Studies 

While policy makers need the kind of information alluded lo above, per* 
sons responsible for the development and implementation of CBTE pro* 
grams need a different kind of information. Obviously, they must also be 
concerned with cost and benefit questions* but the central focus of program 
developers and managers is the effectiveness of alternative program struc- 
tures and procedures m bringing about a specified set of teaching compe* 
tenctes Examples of the kinds of information needed at this level of decision 
making include the costs and benefits of a field based, as opposed to a 
non-field based, mode of operation, the costs and benefits of "simulated" 
field conditions as opposed to "reaMife" field conditions; the costs and 
benefits of implementing an instructional program in "module" form as 
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opposed to a fuf^i !htit rehes upon syllabi, readings, discussions, and the 
like, and the costs and benefits of including m a program elements that deal 
With understanding self as teacher, interpersonal awareness and sensitivity, 
and skills in group problem solving Without such information, decisions as 
to program design and operation must also be based on something other 
than firm Knowledge 

As in the case of pohcy oriented research, two conditions must exist 
before practice oriented research can be earned out First, programs must 
be in operation that reflect the practice of concern. Second, either programs 
must be found that are comparable m structure, organization, and student 
body and are willing to compare alternative practices, or a program must be 
found that is willing to design and carry out alternative practices under 
experimental conditions while continuing to carry out program operations. 

In some respects, these conditions resemble those needed to carry out 
policy oriented studies The existence of fully operational programs is a case 
in point. In other respects, however, practice oriented studies go beyond 
poUcy oriented studies in their demand upon control over unwanted sources 
of variation in this respect they become more difficult to carry out. for 
they impose considerably greater constraints upon programs that are taking 
part m the research 

Given these conditions, practice oriented studies m the arena of CBTE 
Will probably take the form of * systematically manipulated" field expen* 
ments. fn contrast to the natural" field experiments that characterize policy 
oriented studies 

By Viewing the requirements fo" practice oriented studies m this light, 
both policy makers and practitioners should be able to understand why 
research of this kind has not been undertaken thus far m the history of the 
CBTE movement Simply put. it ts for the same reason that policy studies 
have not been undertaken there have not been programs sufficiently well 
developed to permit such research to be earned out Now that CBTE pro- 
grams are beginning to reach maturity, research of this kind becomes pos- 
sible 

Intra-lnftlHutional Studios 

Since the effects of program practices and procedures would be expected 
to interact powerfully with learner characteristic and the kind of learning 
outcomes desired, it is likely that most practice oriented research will be 
carried out withm the context of a particular CBTE program. This will permit 
as many sources of unwanted variation m such studies to be controlled as 
possible The conditions outlined earlier, however, must be obtained before 
a CBTE program would qualify as a site for such research. Paraphrased, 
these conditions would include a fully operational CBTE program where 
personnel of the program are interested in and able to carry out experiments 
on the effects of systematically designed program variations while keeping 
constant the learning outcomes desired from the program. In addition, a 
participating institution must have a reasonably iu^ge number of students 
enrolled m its CBTE program; it must be orga^i?e<^ n such a way that bona 
fide experimental and control groups can be established: it must be 
orgar.ized in such a way that true component variations are able to be im* 
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plemented; and high quality measures of learning outcomes must be obtain- 
able The criterion or dependent measures to be used in such a design, the 
sources of variation to be controlled: and the dimensions of cost to be 
attended to are summarised in Table 3 r 

A note of caution needs to be introduced at this point The fact that prac- 
tice oriented studies require the careful control of unwanted sources of 
variation m both treatment conditions and outcome measures has led edu- 
cational lesearchers historically to undertake practice oriented research 
in a particular way. First, it has tended to be earned out within contexts 
that would "tolerate" such manipulation~-for example, an instructor's own 
class or some limited segment of a student population. Second, it has tended 
to involve definitions of educational practices that were relatively narrow or 
limited in scope, for example, a one day or one week program of instruction, 
a particular grading practice, or a particular set of role relationships between 
instructor and student that are to be tried for a term. Both strategies have 
given rise to the problem of the results of such studies having limited 
uti:iy-~-even withm the context where they were done—and limited gen- 
erahzability 

These are the problems that have plagued educational researchers for 
decades. Only now are we coming to understand what must be done to over- 
come them. Basically they are problems that have to do with the external 
validity of education experiments. Shulman speaks to such problems point- 
edly when he says 

Researchers are caught in a bind. To maximize the interna^ 
validity of experiments, they develop carefully monitored 
settings within which they can govern their research. This has 
long been recognized as a necessity, but it is likely that the 
experimental tradition in American education over-empha- 
sized the importance of reliability and precision at the ex- 
pense of the characteristics affecting that other factor of 
equal importance (external validity) ... It is not sufficient 
that the individuals studied as a sampie are truly representa- 
tivf* of that human population to which the results of a par- 
ticular experiment will be inferred. Researchers must also 
ascertain that the experimental conditions can serve as a 
sample from which to make inferences to a population of 
external conditions of interest. That is. researchers must 
also attempt to maximize the similarity between the condi- 
tions in which they study behavior and those other conditions, 
whatever iney may be. to which researchers may ultimately 
wish to make inferences. The similarity should hold between 
psychologically meaningful features of the settings, not 
merely between the manifest ^spects of the two situations. 
(1970. p. 377) 

If Shulman's analysis is correct, three fundamental shifts m the nature 
and focus of educational research must take place if it is to be responsive 
to the criteria of external validity: 1) a shift m emphasis from the sampling 
of people to the sampling of educational environments; 2) a shift from the 
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Study of Single, isolated variables to the study of the complex of variables 
that make up ongoing educational environments: and 3) a shift from piece- 
meal, unrelated. "one*shot*' research efforts to studies that are articulated 
through time, theory, and problem focus. It is the contention of the research 
committee of the Consortium of CBE Centers that CBTE programs provide 
the Kind of contexts where such shifts, for the first time, have some hope of 
coming about. 

Having said this, it is important to point out that the conduct of long-term, 
multi-faceted, conte/tually valid experimental research lays heavy demands 
on an ongoing educational context, and unless the context is special in many 
ways, such demands wiil not be met. in the judgment of the committee, at 
least five conditions must prevail: 

• Persons responsibie for the mar^agement and operation of the pro- 
grarr must be inclined toward experimentation. Commitment to em- 
piricism and the desire to know must be dominant features of the 
context. Research must be viewed as an integral part of program 
operation, and as such viewed as a continuous, necessary, and de- 
sirable part of the program. 

• Persons responsible for the management and operation of the edu- 
cation program must view the program as Subject to continuous 
change, and view as a major data source for its change a systemat- 
ically designed program of research on its effectiveness. When 
viewed in this way. research can have both the immediate and ap- 
plied value that is needed for it to be supported by those responsible. 

» Data of a quality that will support trustworthy research must be 
collected as a normal part of program operation. Accurate, reliable 
descriptions of program operations must be provided and accurate, 
reliable measures ot learning outcomes must be recorded as a matter 
of course. 

• Sophisticated data management, storage, retrieval, and display capa- 
bilities must be available. Data to be used for research, program 
operation, and program adaptation purposes must be routinely stored 
on computers and routinely retrieved in formats that support program 
related decision making and research. 

• There must be an advisory structure that ensures that the research 
that is pursued has value to persons in the program as well as to the 
profession at large. 

It is recognized that these five conditions represent an unusual array 
of features to find in any ongoing educational context today. It is also recog- 
nized that they are difficult conditions to bring about if they are not already 
in existence. It is the judgment of the committee, however, that CBTE pro- 
grams represent the most hopeful development in the history of education 
for such conditions to be met. 

inttrlnstnutional Studies 

While most practice oriented studies will be carried out within a particular 
institution, at least initially, the continuing concern for the utility and gen- 



id 

ERLC 



DCST CC?Y AVAiUBLE 

eraliiability of the results ot such studies requires that they be replicated 
ultimately m a variety of educational contexts To some extent, this require- 
ment can be met by testing a particular practice m different programs, or 
With different populations of students withm a single program in the long 
run. however, this is not sufficient, for until it can be shown that a particular 
practice is successful m a number of institutions, it is not likely that it w !' 
have a great deal of acceptance m or utility to te«icher education at large. 

In carrying Out mtennstitutionai studies on alternative practices, it will 
be necessary of course to control for the same sources of variation and to 
take the same measures of cost that are outlined m Table 3 2 



Part III: Basic Studies 

Historically, research m education and teacher education has been dis- 
appointing (Gage. 1963; Smith. 1972; Travers. 1973). A host of explana- 
tions have been offered for this fact, including the argument that the world 
of education is simply too complex to yield to the strategies of science, but 
two explanations seem to have most authority. The first is the failure of 
education research to meet the requirements of external validity. 
The second is its lack ot adequate dependent or outcome measures. 
No matter how powerful a research design, or how powerful a treatment 
condition, research results of significance will not be forthcoming unless 
the measure of effect is strong. 

One of the most hopeful features of the competency based education and 
teacher education movements is that they are designed specifically to over- 
come the perennial problem of outcome measurement. If carried to suc- 
cessful completion, competency based education should yield new and 
powerful measures of learning outcomes m pupils, and competency based 
teacher education should yield new and powerful measures of teacher per- 
formance. If these measures are ot the kind and quality anticipated, and 
the research community recognizes them as such and takes advantage of 
their availability, basic research m education and teacher education should 
profit immensely 

Two ccndiiions must be met. however, if competency based education 
and teacher education programs are to become contexts for basic research. 
First, as implied m the above paragraph, the measures employed m the 
assessment of teaching competence must be of a quality that permits their 
use m research. That is. they must be valid, reliable, and sensitive. Second, 
experimental designs must be employed m the context of program opera- 
tion With sufficient rigor that causality" can be attributed to the experi- 
mental or treatment vr^nables investigated Both of these conditions are 
above and beyond the requirements of normal program operation, but both 
can be achieved if introduced with care and foresight. When both are met, 
much IS to be gamed, for basic research can be earned out as an adjunct 
to normal program operation at httle added cost. 

While It IS possible to combine basic research with program operation 
by meeting these two conditions, it needs to be pointed out that considerable 
risk IS involved in attempting such a venture. High quality measures, for 
example, are often difficult and costly to obtain. Also, requiring that pro- 
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gram operations meet the car^straints of experimental design almost always 
creates a cumbersomeness and rigidity that frustrates program managers 
and participants. Heretofore efforts to design data collection systems that 
support both program operation and basic research have tended to end in 
the design of research programs instead of operational programs that have 
good data. When this has occurred, there has been a nearly universal reac- 
tion on the part of program managers and participants throw the research- 
ers out! (Parlettand Hamilton. 1972). 

If th!S pitfatt IS recognized, it is still possible that, if done with care, data 
generation systems can be designed that will support both program opera^ 
tton and basic research When this is the case, the best possible context 
for basic research exists it can be carried out at low cost, and it has a good 
chance of meeting the requirements of external validity that are not met in 
most educational experiments. 

Notes On T»rminology 

While the distinction between basic and applied research has never been 
particularly clear (Cohen. 1948: National Science Foundation. 1960: 
Brooks. 1967) the distinction has been maintained for purposes of the 
presera paper Following the proposals of Schalock and Sell (1972). basic 
research r n instruction has as its focus (a) the identification and delineation 
of the constructs central to instruction and (b) the establishment of pr/n- 
c/p/es of instruction that incorporate these constructs. Applied research, on 
the other hand, has as its primary focus the testing of instructional practices, 
which may or may not reflect the principles of instruction developed through 
basic research. When viewed in this way. both the policy and practice 
oriented studios described in the previous sections of the paper are seen as 
classes of applied research. They have direct and Immediate utility to 
practitioners and policy makers. Typically, basic research has little direct 
utility to the practitioner. Its primary utility is to the applied researcher, 
particularly the applied researcher who is concerned with practice oriented 
studies. 

Another distinction that has never been particularly clear, but one that 
IS central to the present paper, is the distinction between learning and 
teaching. As strange as It may seem. It is a distinction long ignored by 
educational and psychongical researchers (Gage. 1963) but one that in 
recent years is beginning to bear fruit (Nuthall and Snook. 1973: McNeil 
and Popham. 1973). As used In the present paper, theories of learning deal 
with how an individual takes In. stores, transforms, and utilizes information: 
theories of teaching deal with how an individual influences that process. 
In drawing this distinction elsewhere. Schalock has pointed out that 
In a sense the distinction is arbitrary, for the end point of 
both learning. In another sense, however. It is not. for the 
focus of one is upon the processes of learning and the other 
upon the conditions of learning. By forcing the distinction, 
and then attending systematically to the conditions of instruc- 
tion. Issues that tend to be obscured when focusing upon 
learning come into full view; for example, the dependency 
of instructional decisions upon educational objectives and 



ERLC 



BEST COPY AVAILABLE 



learner characteristics In their pursuit of laws governing 
learning. expenfT)ef)tal psychologists have not attended sys* 
tematically to either of these classes of information . The 
assumption underlying the present effort is that by highlight* 
ing the instructional process, and by attending to »t both 
conceptually and empirically, it will be possible »n time to 
develop a productive science of instruction which, in turn, will 
permit an effective and efficient technology of instruction 
(1972. p 39) 

A Paradigm For Basic Research On Teaching 

Without wishing to become embroiled m the paradigm-model-theory de- 
bate (BrodbecK. 1963. Gage. 1963. Snow. 1973). it is necessary to present 
the conceptual framework that underlies the proposals to be made Whether 
the framework is a model, a paradigm, or a first cut at a C- or D-Theory is 
unclear. What is clear are the direcuonal implications of the framework 
for research on teaching. Smce these directional implications are im- 
modestly seep as " defining the legitimate problems and methods of a 
research field for succeeding generations of practitioners" (Kuhn. 1962. 
p. 10). the framework is equally immodestly referred to as a paradigm. 

The paradigm proposed as a guide to basic research on teaching reflects 
a point of view about teaching that is not inconsistent with how teaching is 
viewed generally, but it is inconsistent with how most research on teaching 
has been done The paradigm can best be described as a " context depen- 
dent" paradigm, for it is based on the assumption that for teaching to be 
effective, what a teacher does must always be adjusted to the nature of the 
learning outcome(s) desired, the nature of the learner (s) being taught, 
the nature of the setting within which learning-teaching is occurring, and 
whatever is known about how learning occurs As such, effective teaching is 
seen as representing an articulation of a remarkable nufnber of variables, 
especially when a wide variety of learning outcomes are bemg pursued by 
a wide variety of learner types under a wide variety of learning conditions. 
The paradigm is portrayed schematically as Figure 3 2 

Two comments need to be made about the paradigm. First, it has strong 
directional implications for research on teaching. It specifies the major 
classes of variables to be attended to in such research, and it specifies 
that these variables be treated in designs that accommodate them as inter- 
dependent or contextually dependent" variables. 

Second. It lends perspective to the anxiety that many educators feel about 
the continued emphasis m CBTE on teacher behaviors and the continued 
press to carry out research that attempts to link teacher behavior to pupil 
outcomes (see articles by Rosner and Kay. McDonald, and Flanders in the 
January. 1974. issue of the Phi Delta Kappan as examples of the continued 
press for such research) Withm the framework of the proposed paradigm, 
teacher behavior may be treated as either a dependent or an independent 
variable, and when it is treated as an independent variable it must be done 
so withm a design that links it to student characteristics, setting character- 
istics. Class of learning outcome pursued, and the learning processes in- 
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FACTORS TO BE CONSIDERED IN ESTABLISHING 
THE CONDITIONS OF INSTRUCTION AT A GIVEN 
POiNT IN TitViE 



Desired 




Learning 



(The Circles on the arrowed lines represent learner be- 
havior and the squiggles on those lines represent the 
filtering effect of teacher characteristics on the perception 
of learner behavior and the factors affecting learning.) 



volved m the mastery of such outcomes. 

Within such a framework, the continued search for simple correlations 
between teacher behaviors and learning outcomes m children does not 
carry much appeal. Nor does it carry much hope, considering the near 
infinite array of permutations possible between particular behaviors, par- 
ticular learnings, particular learning outcomes, particular teachers, etc. 
The conclusions reached by Rosenshine and Furst (1971). therefore, should 
not be Surprising. Nor should the kind of criticism that Maxwell has recently 
leveled at persons responsible for the design of CBTE programs who con- 
tinue to define teacher competence m terms of specific teacher behaviors. 

We don t have a list of the competencies of the successful 
teacher for the simple reason that teaching is an art— not a 
science. Behavior that is a valuable asset to a given teacher 
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with a certain personality in many (but not all) situations, in- 
sofar as most (but not all) of his pupils are concerned, may 
well go over like a lead balloon (to most students) if affected 
by some other teacher with a different personality. We can't 
even say that a teacher should love his students, since many 
teachers that we and the students consider successful exhibit 
no behavior (measurable or otherwise) thdt would lead one 
to this conclusion. 

The teacher/student relationship is one characterized by 
heterogeneity on both sides of the relationship. The art lies in 
knowing, or sensing, what particular stimulus or behavior is 
best for me as an individual teacher to reach most effectively 
a particular student or group of students in a given set of 
circumstances. To assert that the "competent * teacher "will 
demonstrate his concern for persons in the lower socio* 
economic groups by voluntarily spending at least two hours 
per week working in a social agency" is absurd, no matter 
how highly we value such concern or approve of such work. 
We CdO make no such simple statement about the competent 
teacher (1974. p 310) 

Reteiurch On Conslructo That Relate To inetructlon 

In order to carry out research on instruction that meets the requirements 
of the paradigm outlined above, it is necessary to define, establish the 
parameters of. and make operational the major variables contained within 
it. The research and development involved in this process is perhaps the 
most basic of all activities that goes on in any science. It is a highly con- 
ceptual activity: it involves basic methodological work in that constructs 
must carry operational definitions and be subject to measurement in order 
to be incorporated into a research program; and it involves a major empiri- 
cal effort m order to determine whether the constructs identified are Suf- 
ficiently powerful to warrant inclusion in research on the paradigm proper. 
In its more advanced stages it is also likely to involve theory testing, since 
any set of constructs take their definition within a theoretical system of one 
kind or another. 

For all these reasons, it is proposed that construct identification and 
measurement be considered as a necessary part of any basic research pro- 
gram on instruction and that it be seen as paralleling or preceding research 
on principles of instruction, that is. research on the relationships between 
the sets of variables outlined in Figure 3 2. 

The design of construct identification and delineation studies assumes 
two basic forms The first is a classic, single variable or comparative re- 
search design where the purpose is to determine whether a particular con- 
struct has any identifiable impact upon learning outcomes in children or 
has any likelihood of such impact even though there is no empirical evidence 
to that effect Research of this kind is always guided by the paradigm that 
reflects the particular view of instruction that is being researched, for exam- 
ple, an "inquiry" or a " precision" approach to instruction. The specific pur- 
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pose of such research is to define withm the broad classes of variables iden- 
tified Within Figure 3.2 specific variables that are promising for inclusion in 
the next level of research, that is. research that seeks to establish laws or 
principles. 

A second research design that is a necessary feature of construct iden- 
tification and delineation studies is a testing of methodology" design. When 
new constructs are identified, or when existing constructs are refined, a set 
of measurement operations have to be developed in order to test the power 
of the constructs. Since the results of any research Study are only as good 
as the measures of the constructs employed within it, the development of 
strong measures for all constructs to be investigated becomes a critically 
important activity. Central in this activity are comparative studies under- 
taken to determine the relative power or effectiveness of alternative ap- 
proaches to the measurement of a particular construct. 

Research of this kind, of course, borders upon and at times may become 
synonymous with the reliability and vahdUy studies that must also be carried 
out as a part of methodological development. 

As in the case of validity studies, the issue of criterion is paramount In the 
design d comparative studies of methodology. On what basis is the decision 
to be made that one approach to measurement is superior to another for a 
particular variable or construct? The correlation of one measure with 
another that is known to be a 'valid" measure of the construct under con- 
sideration (the criterion used in "concurrent" validity studies) is an appro* 
prtate strategy, for it provides no information dt all as to which of the two 
measures is in fact the more powerful. To obtain data on the relative power 
of a measure, one has to move to the arena of prediction. As tortuous and 
convoluted as predictive evidence often becomes, it is the only evidence 
that can be looked to with confidence m tests of the comparative power of 
alternative methodologies. Other things being equal, the measure that 
predicts with most accuracy or power must be viewed as the bexxer mea- 
sure (Schalock. 1969). 

In dosing the discussion of construct identification-delineation studies 
and the methodological studies that are a necessary part of such research, 
four observations seem appropriate. First, studies of this kind do not need 
to "start from scratch. ' Large literatures exist with respect to at least four 
of the five clusters of variables identified in Figure 3.2. and a literature is 
beginning to emerge m relation to the fifth. Second, research of this kind has 
not. and need not. be carried out only by educational researchers. Biolo- 
gists, psychologists, anthropologists, and sociologists are all involved in the 
identification and delineation of constructs that contribute to the conceptual 
structure called for by the paradigm. Third, the process of construct identi- 
fication and delineation is laborious, complex, and time consuming. It is 
also never ending, for implicit m the philosophy of science is the assumption 
that constructs will never cease to evolve nor ever find their absolute defi- 
nition. In this context, it is probably safe to say that several hundred life- 
times of research will be needed on each of the classes of variables identi- 
fied in Figure 3.2 before constructs will surface that are sufficiently powerful 
to permit research in education to proceed with the hopefulness of having 
an effect which research carries in the basic sciences. 
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Finally, the same re^isons that m«iHe a CBTE conteKt particularly desir* 
able for carrying out practice oriented studies are the same reasons that 
make tt a particularly attractive context for construct identificatton-delinea* 
tion studies It a stable, continuously available research context; much of 
the data needed for such research are available as a consequence of nor- 
mal program operation, thus making it a tow-cost research context; and 
the variables being investigated are the real-life variables with which teacher 
educators and educators must deal Stability, accessibility, low cost, and 
the likelihood of being able to overcome the problrri of the external validity 
of experiments constitute a lot of value to be gamed. 

Rmew'ch on PrInciplM of inttruetion 

When construct identification-delineation research has progressed to the 
point where reasonable confidence can be placed in one or more variables 
withm the various classes of variables specified m Figure 3.2. research on 
the identification of instructional principles can begin. As used m the present 
paper, a principle of instruction i$ an empirically established relationship 
between a particular set of instructional conditions and a particular class 
of learning outcome for learners who have a particular set of learning* 
related characteristics and are working under a particular set of learning 
conditions The power of such principles rests in the guidance they provide 
for on»line instructional decisions made by teachers, and the more reflec* 
five decisions made by the designers of instructional materials and pro- 
grams. If the empirical relationships established are strong, teachers and 
the designers of instructional materials wHi have a set of guidelines that 
for the first time will enable them to establish instructional conditions and 
be reasonably sure that those conditions will make the kind of difference de* 
sired. Until such empirical relationships are estabhshe^l. instruction and 
the design of instructional materials must of necessity be an iterative pro* 
cess of best guesses and successive approximations. 

In some respects, the design of research that aims toward the identifica- 
tion of principles of instruction is more complex than that aimed at the 
identification of constructs and development of methodology, but m other 
respects It IS less so On the one hand. principle*related research requires 
the use of designs that will accommodate a large number of sources of 
variation m the course of a sirtgie study, whereas construct related research 
does not On the other hand, the criterion problem is less troublesome in 
principle*related research than it is in construct related research. The cri- 
terion of progress m relation to desired learning outcomes is straightforward 
and unambiguous, whereas the criterion of predictive utility is anything but. 
Be this as it may. there are designs and methods of analysis that can ac- 
commodate the complexity required by principle related research. Proto- 
typic m this regard are the designs and analyses used in aptitude-treatment 
interaction studies <Bracht. 1970; Cronbdch and Snow. 1969). 

As in the case of construct related studies, a number of brief comments 
need to be made before leaving the discussion of studies aimed at estab- 
lishing instructional principles. First, studies of this kind do not need to 
start from scratch. ' but they do not have the conceptual and empirical 
base en|oyed by construct related studies. Aptitude-treatment interaction 
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Studies are prototypic. but they need to accommodate the additional vari- 
ables identified m Figure 3.2. Second, research of this Kind will probably be 
undertaken only by educational researchers While biologists, psychologists, 
sociologists, and anthropologists search for and work with many of the same 
constructs that need to be involved m instructional research, they will rarely 
search for instructional principles of the kind considered here. Third, the 
process of establishing instructional principles is just as laborious, com- 
plex, and time consuming as the process of delineating constructs and 
developing methodology, it is also as neverending. In part, this is because 
of the interdependence of construct and principle related studies — as 
better constructs are established, better principle-related research can be 
undertaken, as better principles of instruction emerge, sharper constructs 
and finer measures can be developed in part, however, it is because of the 
implicit assumption in the philosophy of science that principles once es<> 
tabhshed will forever be refined. 

Finally, the reasons that make a CBTE context a particularly attractive 
context for all of the other kinds of research that have t}een discussed are 
the same reasons that make it a particularly attractive context within which 
to pursue research leading to the establishment of instructional principles. 
It IS a stable, continuously available context where the cost of carrying out 
research is low and the fidelity (external validity) of the variables high. 
Given the likelihood of a large number of "pilot" or "exploratory" studies 
being performed ith<% frequency of false leads and wrong predictions will 
undoubtedly be high m the beginning) and the requirement of a large num<^ 
ber of replications to establish the confidence levels needed if practitioners 
are to rely upon the emerging principles as guides to instruction and instruc* 
tiondl development, a research context that has CBTE-like properties would 
seem to be a necessary condition for the conduct of principle-related re- 
search. Whether ongoing CBTE programs can be found that will tolerate the 
demands placed upon them by such research, and at the same time possess 
staff with the competence to carry out such research, is an open question. 
Until Such contexts are planned for. res^^^rch that makes an appreciable 
difference m instructional design and prac: ^0 is not likely to occur. 

Part IV: Documentation Studtta 

It was argued earlier m this paper that CBTE needed to be the focus of 
documentary or case studies as well as basic and applied research studies. 
Competency based programs are a great deal more complex in their struc* 
ture and organization than traditional programs and require for their devel* 
opment and implementation a level of assistance that new programs out of 
a traditional mold do no* The point of view adopted in the present paper is 
that an extremely useful way to provide such assistance, and indeed per* 
haps the only way. is through detailed documentation of the development 
and operation of existing CBTE programs Knowledge of the development 
and implementation process in CBTE represents as much a knowledge gap 
as the knowledge gaps to be closed through the research directions that 
have been outlined At least three kinds of case studies are needed within 
which both developmental and operational issues need to be attended to. 
One kind of case study needs to focus on program development and opera- 
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tion. another one s»olected «)Hpects of CBTE programs that are particularly 
complex to develop and operate, or that have been particularly tardy in their 
development, and still another on the diffusion of program practices within 
an institution or to other institutions within a state or region. Documenta- 
tions of the first Kind would require finding programs reasonably wetl along 
in their development that reflect m their operation all of the basic principles 
called for m the generic conception of competency based teacher educa- 
tion Documentations of the second kind would require only that the aspect 
of program operation to be documented be under development Examples 
of program elements that might be the focus of second level documenta- f 
tions include selected consortium or coalition arrangements, selected ap- k 
preaches to competency definition and assessment, and selected ap- \ 
proaches to program adaptation on the basis of program and student per- ^ 
formance data 

Documentation of the third kind would require the identification of pro- 
grams that could point clearly to program related practices or procedures 
that have been adopted by others This type of documentation would allow 
for Identification of diffusion related activities that could account for the 
adoptions 

Collectively, these various approaches to documentation would give the 
potential developer of CBTE programs access to critically needed infer ma* 
tion. They would provide access to strategies and procedures that have 
tseen used to develop such programs or to make such programs available 
to others They would also provide some sense of the effectiveness of such 
strategies and procedures, in short, jt carefully selected, they would pro* 
vide much of the information needed to select from among options to pro- 
gram development, assist in carrying out program development, and sug* 
gest means for helping others adopt procedures and practices tested 
through program development. All are conditions that must exist if the 
principles of CBTE are ever to be adopted widely. 

Within all three kinds of documentation studies, attention should be given 
to both developmental and operational issues. It is further proposed that, 
withm both developmental and operational foci, attention be paid particularly 
to f^e critical decision points encountered, the decision alternatives con- 
sidered. the decision choice made, the conseguenees of the decision made 
and, to the eMtent possible, the likelihood of the consequences of alternative 
decisions considered, it is information of this kind and at this level of detail, 
especially as it reflects the reality of real life operational constraints, that is 
most critically needed by those who wish to initiate or dif^seminate CBTE 
programs 

On the basis of extended experience m preparma case studies of ongoing 
research and development programs (Schalock. et. al.. 1972). it is essen- 
tial that the preparation of documentary studies be undertaken with as much 
planning, as much forethought, as much professional expertise, and as large 
a resource base as any of the applied research studies that have been called 
for. it IS unclear, however, how best to actually carry out such studies. One 
procedure would be to contract with an agency to carry out all documen- 
taries This would involve the preparation of staff, the collection of data, 
the provision of case study reports, etc. The advantage of such a procedure. 
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as compared to a senc^s o( i»idcpendently initiated studies. c»^» efficiency, 
comparability of reporting tormats. and reasonably good assurance of 
quality The disadvantage is the lack of firsthand familiarity with the data 
to be reported and tNe aependence upon the recall of others to provide such 
data. 

An alternative approach would be to provide only for the central coordi* 
nation and management of the documentation effort at the Federal level, 
for example, establish specifications as to substance, format, etc.. and tram 
staff from the institutions whose programs are to be reported and have 
them serve as scribes or reporters of the development and implementation 
process as it is occurring Either procedure would require a considerable 
funding base, a great deal of preparatory and design work, access to exper- 
tise in case study preparation, and a well designed management system to 
insure optimum quality at minimum costs Without a ctear recognition of the 
complexities involved m the preparation of the kind of case studies needed, 
there IS danger that an effort to prepare such studies will be underfunded 
and understaffed. 

A by-product of the documentation studies outlined, or perhaps a com- 
panion to them. 1$ a series of case studies that document the development 
and operation of CBTE programs that serve as contexts for research. While 
relatively few CBTE programs will meet the criteria needed to function as 
research centers, it would seem to be important to include them in the 
overalt documentation effort Given the promise that such centers hold for 
educational research and given the nature of the relationships that must 
exist between instruction and research if a program is to serve as a func- 
tional research context, it would seem that documentaries of this kind 
would be of particular interest to an agency such as NiE. Since the creation 
of CBTE centers for research may be to education what the creation of 
laboratories was to tbe physical sciences, agencies such as NIE should 
understand all they possibly can about how such centers function. 

Part V: ProMems« Protpecis, and a Plan of Procadure 

Outlining the kind of research and development program needed m rela- 
tion to CBTE IS one thing Implementing what has been outlined is something 
else. It seems appropriate, therefore, to close a paper of this kind with a 
brief look at the task of translating what has been proposed into practice 
and to offer at least tentative observations as to how that translation process 
might take place. 

Some Antlcloated ProMemt 

It has been argued that only now ib it possible to begin to undertake Uie 
kind of research and documentation studies that have been called for in the 
paper because only now are functional CBTE programs beginning to t)e 
established, it is smply not possible to compare the cost and benefits of 
CBTE with non-CBTE programs until CBTE programs are available for com- 
parison. The same argument obviously pertains to using CBTE programs 
as contexts within which to carry out practice oriented research, basic 
research, and documentation studies 
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Even though illustrattve CBTE programs are now beginning to emerge* 
a host of problems remain before the kind of research and documentation 
called for in the present paper can be undertaken. Many such problems 
are referred to in the course of the paper; for example, finding CBTE pro* 
grams that are willing to face comparison with other programs or that are 
wilting to accommodate the demands of ongoing practice oriented and basic 
research studies. Other problems remain, however, and need at least to be 
identified. The more important of these would seem to be: 

• Assuring the continued development and operation of competency 
based teacher education programs as demonstration and research 
contexts; 

• Securing the funding level needed to carry out the research and docu* 
mentation studies outlined in the present paper; 

• Finding competencv based teacher education programs that are at a 
level of development and possess sufficiently competent staff to 
carry out the kind of research and documentation programs outlined; 

• Fixing responsibility for the long term coordination and management 
of the research and documentation program called for. 

A numUer of these problems revolve around the availability of funds. This 
is obviously the case with respect to the research activities that have been 
proposed. Funding is also involved. Lowever. in the continued development 
and operation of CBTE programs. Unless viable CBTE programs exist, most 
of what has been proposed is. in the parlance of the day. inoperative. 

Some of these problems are not so much a matter of money as they are 
the availability of personnel who are competent to carry out the functions 
that will need to be carried out. In this regard, two broad sets of functions 
need to be performed — developing and maintaining a CBTE program that 
can In fact support the kind of research that has been called for; and 
actually carrying out the research and documentation activities. 

Finally, a number of these problems are plainly and simply a matter of 
the National Institute of Education and the U. S. Office of Education agree- 
ing to the importance of the kind of research and development effort that 
has been outlined; the probability of its short* and long-term contribution; 
the priorities of the proposed program as compared to the priorities of other 
programs that are being supported; the priorities of the various activities 
called for within the overall program; and the responsibilities to be assumed 
individually and collectively by the two agencies for the implementation* 
funding, cind management of the program. In approaching the various 
problems and issues that have been identified, it is recommended that the 
last issue mentioned be attended to first, since decisions reached at that 
level will largely determine what happens in relation to all else. 

Some Opllmiillc Pro|ecliont 

Assuming that the National Institute of Education and the U. S. Office 
of Education were able to reach the decision that the proposed program was 
worth Supporting, that it had high priority in relation to other programs, 
that agreements were able to be reached in terms of shared responsibility 
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for supporting and managing the program, that a reasonably high level of 
funding was made available to the program, and that operational CBTE 
programs could be found that were willing to serve as research contexts. 
What could be expected to emerge from the program at the end of a 3-year 
period? at the end of 5 years? at the end of 10 years? The projections that 
follow attempt an answer to these questions They assume that all lines of 
inquiry that have been recommended are to be supported 

Products And Conditions At 
Ttie End Of Three Years 

DATA FROM POLICY ORIENTED STUDIES 

• Preliminary data on costs and benefits of CBTE and non-CBTE pro- 
grams 

• Preliminary data on costs and benefits of alternative forms of CBTE 
programs 

• Methodologies, settings, and personnel established to carry out long- 
term policy studies 

DATA FROM PRACTICE ORIENTED STUDIES 

• Preliminary data on the costs and benefits associated with selected 
instructional practices within particular CBTE contexts 

• Methodologies, settings, and personnel established to carry out long- 
term studies of the costs and benefits associated with selected in- 
structional practices within particular CBTE institutions 

• Methodologies, settings, and personnel established to carry out 
cross-institution replications of cost and benefit studies on selected 
instructional practices 

DATA FROM BASIC RESEARCH STUDIES 

• First level taxonomic work completed in relation to the construct 
delineation studies, related literature reviews completed, design and 
measurement methodologies tested, and some replication data on 
constructs that have appeared promising in existing education and 
psychological literature 

• Theoretical positions that link the const<"icts identified in the research 
paradigm stated at an initial level of c .ail. related literature reviews 
completed, design and analysis methodologies tested, and studies in 
progress that test the soundness of the theoretical structures that 
guide the assembling of variables 

CASE STUDIES FROM THE DOCUMENTATION EFFORTS 

• At least three case studies on selected CBTE institutions covering 
at least two years of development and implementation decisions 

• At least SIX case studies on selected attributes of competency based 
teacher education that have 2-year history of development and im- 
plementation 
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The End of Pfve Years 

DATA FROM POLICY ORIENTED STUDIES 

• Sufficient data on costs and benefits of CBTE and non-CBTE programs 
that policy decisions can be made with reasonable confidence 

• Sufficient data on costs and benefits of alternative forms of CBTE 
programs that policy decisions can be made with reasonable con- 
fidence 

• Meihodologies. settings, and personnel established to carry out 
whatever additional policy studies are needed 

DATA FROM PRACTICE ORIENTED STUDIES 

• Sufficient data on the short*term costs and benefits associated with 
selected instructional practices m particular CBTE contexts that 
program implementation decisions can be made with reasonable 
confidence 

• Preliminary data on the long-term costs and benefits associated 
with selected instructional practices in particular CBTE contexts 

• Initial replications across institutional settings of cost/benefit studies 
on selected instructional practices 

DATA FROM BASiC RESEARCH STUDIES 

• An initial set of empirically grounded constructs within each cluster 
of variables identified in the research paradigm that has been pro- 
posed 

• Measurement methodologies firmly established for each construct 
within each cluster of variables in the paradigm 

• The theoretical structures that link the constructs identified in the 
research paradigm reasonably well worked out and reasonably pro- 
ductive m terms of hypothesis generation 

• Design and analysis methodologies firmly established for research 
on principles of teaching, given the constraints of the paradigm 
governing principle-related research 

• A beginning collection of empirically based principles of instruction 
that are suggestive of the nature 01 the interactions between the 
variables listed in the paradigm on teaching 

CASE STUDIES FROM THE DOCUMENTATION EFFORTS 

• Completed case studies on three selected CBTE institutions (each 
case study would cover a 4-year period of time) 

• Six additional case studies of selected attributes associated with 
competency based teacher education programs that have had at 
least a 2 year history of development and implementation (making a 
total of 12 such documentaries) 

• At least three case studies covering a 2*year period of time on insti- 
tutions that are serving as centers for research in CBTE 
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The End Of Ten Years 

DATA FROM POLICY ORIENTED STUDIES 

• The completion of whatever cost/benefit studies are needed on CBTE 
as opposed to non-CBTE programs 

• The completion of whatever cost/benefit studies are needed on al- 
ternative forms of CBTE programs 

DATA FROM PRACTICE ORIENTED STUDIES 

• Short-term cost and benefits studies essentially completed on se- 
lected instructional practices within particular CBTE contexts 

• Sufficient data on long-term cost and benefits of selected instructional 
practices m particular CBTE contexts that program decisions can 
b«r made with reasonable confidence 

• Sufficient cross-mstitutional replication data on cost/benefit studies 
of selected instructional practices that program decisions bv a wide 
variety of institutions can be made with considerable confidence 

DATA FROM BASIC RESEARCH STUDIES 

• Second generation construct delineation research underway and 
the theoretical structures within which constructs rest under empiri- 
cal test 

• Second generation design and methodology in relation to construct 
delineation studies under development 

• A sizftable number of principles of teaching established, and estab- 
lished with sufficient confidence that program design as well as on- 
line teaching is able to benefit from their application 

CASE STUDIES FROM THE DOCUMENTATION EFFORTS 

• All case studies on CBTE programs and program elements com- 
pleted 

• All case studies on CBTE programs as contexts for research com- 
pleted 

Figure 3.3 repiesents a translation of these projections into a simple time/ 
task Chart. 

A Pt«n Of Procedure 

Any of a dozen strategies could be followed in implementing the pro- 
gram of research and documentation that has been proposed. Each would 
have advantages and disadvantages; and more than one would undoubtedly 
prove to be successful Be that as i* may. a number of recommendations 
are offered at this point to assist in its implementation. The recommenda- 
tions assume NIE and OE agreement as to joint sponsorship of the pro- 
posed program and agreement as to the division of responsibilities therein. 
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1. Begin work on all aspects of the research proposed, even though 
there may not be enough money to do so on a large scale. All of the 
research efforts outlined need to be done; each will contribute to the 
other; and each will serve the needs of different audiences. 

2. Take fully into account the program development work needed in 
order to carry out the research that has been proposed, and then see 
to it that the development and the research aspects of the overall 
program are closely coordinated. Without such attention to develop* 
ment there is danger that the research contexts on which the entire 
proposal is predicated will not be available. 

3. Share the research and development costs across Federal agencies; 
for example, have NIE assume the costs of research and OE assume 
the costs of development and documentation. 

4. Keep responsibility for managing the overall research program a 
Federal responsibility, rather than a state or non-Federal responsi* 
bility. 

5. Hinge the program on competitive proposals that are in response to 
RFPs. 

6. Direct most of the funding for research to institutions that have 
CBTE programs that can function as Research and Development con- 
texts (funds should not be directed to existing Research and Develop- 
ment Centers uniass they are linked to CBTE programs that meet 
the criteria for CBTE research centers). 

7. Before letting research contracts to any institution, establish care- 
fully the characteristics neecfed by an institution to carry out the 
various research and development activities that have been called 
for. 

8. Implement a dissemination function of major proportions, and target 
the research and documentation results to be disseminated to the 
audiences for which they are particularly appropriate. 

Obviously, the research program proposed is a massive undertaking. It 
is also an extremely complex one, and one that in the minds of many would 
be too slow to produce results or too grandiose ever to get started. In the 
judgment of the members of the Consortium of CBE Centers, however— and 
It is a judgment that has been carefully weighed— it is precisely this kind of 
research program that is needed in order for teacher educatiori programs 
ever to make the kind of difference in teaching that they are supposed to 
make. The irony and hope is that by using CBTE programs as contexts for 
research, costs will be sufficiently reduced and the problem of the external 
validity of experiments will be sufficiently overcome so that the program 
that has been proposed can be carried out successfully. 
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Bruc» R. Joyct 

The purpose of research on teaching effects is to identify teaching skills 
which induce learning of different l<inds. Therefore, it is prescriptive re- 
search in the sense that it seeks relationships which can guide teachers, 
teacher trainers, and pohcy makers. By now most of us recognize that valid 
prescriptions will be developed only slowly and after intensive cumulative 
research programs have been established for some time. These programs 
will gradually clarify the outlines of skill in teaching and relate them to 
growth by learner «. What we will find depends largely on what we learn to 
look for. 

In this paper. I seek to clarify several issues which affect the shape of 
such research programs — their direction and their technology. 

Some of these issues are technical and some are philosophical. We begin 
with a philosophical issue with considerable impliCfi*ions for the shape of 
Our research. 

The Teacher and the Learner: WHh Whom do we Start? 

I firmly believe that we have to begin with the conception of the learner 
rather than a conception of teaching. I think we are in some danger of for- 
getting one of the most important messages that we can get from John 
Dewey s attempt to define the process of education.^ Specifically, I am 
referring to the belief that learning is a product of the interaction of the 
learner with an environment. The importance of this definition lies in the 
emphasis on the learner s activity and the perspective that the educator 
facilitates learning by creating environments. 

When we begin with the conception that it is the student s own activity 
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that results m letirmng ,'inc1 that the role of the educator lies tn the provision 
of the environments with which the learner interacts, then the teacher be- 
comes important as he contributes to. and is part of. an environment He is 
obviously only one of the contributing agents, but he is a critical one His 
role broadens from an instructional one to the multidimensional set of roles 
that contributes to the creation of the environment (The teacher helps to 
buiid the social system, to develop a sense of rapport among the students, 
to bring resources together, to help the learner focus his activity, to set out 
tasks, and so on ) Teaching sKill from this perspective is the creation of 
environments. This conception of teaching has a powerful effect on the 
search for sKill and an even greater effect on the evaluation of the effects of 
a skill, for a skill appears as part of a complicated environment with which 
a complex bundle of sensations and capacities — the learner — interacts. 

To begin with, we do not expect the effects of an environment to be 
mechanical and simple. A change m environment will ordinarily be ex- 
pected to have only a gradual effect on the learner He has to adjust to the 
environment, and if it is very different from previous ones, the adjustment 
usually takes some time. In addition, because it is the learner s activity that 
results in learning, hts motivation, intelligence, experience, and skill will 
naturally be very powerful determinants of what results We do not expect 
that environments will have a mechanical, homogeneous effect on any 
group of learners that interacts into them. At times, as I read some of the 
literature on competency based education and review some of the studies 
that are presently gomg on. I feel that the teacher is being conceived almost 
as a swordsman who lays about the learner with his repertory of sharp- 
poin.ed skills, each of which will have a mechanical effect on learning out- 
comes The expectation seems to be that if he pushes on the learner with a 
certain kind of Question, then a certain kind of learning will pop out. and we 
Will be able to see it quickly and clearly. Such is not the nature of the inter- 
action between learning environments and learners. It is very rare that any 
one skill (any smgle sword thrust) or any stylistic characteristic will stand 
Out as the Cf.use of the learning Even where we find that increased learning 
IS associated with the presence of a certain kind of skill or style in teaching. 
It IS very likely that the reason is because that skill or aspect of style is an 
index of a much larger complex of behaviors that signals the presence of a 
certain kind of environment (For example, environments characterized by 
prdise are believed by many to produce greater learning, certainly greater 
self-esteem. The skill of praising helps to create such an environment, but 
It IS not very likely that it stands alone. A teacher who authentically and 
warmly praises his learners more than another teacher probably also does 
many, many other things as well. You do not simply turn on praise m a 
matrix of hostility and bring it forth as an unintegrated skill that shines out 
and determines what the learner will do ) 

Teaching, thus, is a multidimensional act that fits m a multidimensional 
context to produce multidimensional environments which are interacted with 
by multidimensional learners Given this condition, research on teaching 
effectiveness will produce a multivariate kind of truth. We will not end up 
With formulas that we can state m unequivocal, mechanical terms such as 
"teaching skill such-and-such increases such-and-such a kind of learning 
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in such-and-such a way A *eachmg sKiH will be more iike'y a small but 
useful part of a certain kind of environment which will be reacted to differ- 
entially by learners and will have a number of kinds of mam and side effects 
It IS because of the really great compienity of the learner-teacher rela- 
tionship that the study of teaching effectiveness appears to be a stab at a 
seamless web To develop a cumulative program of research to identify and 
validate teaching competencies, a number ot issues have to be faced. Some 
of these issues are philosophical* and some are technical, in this paper, 
we Will discuss several of them and suggest how the<r resolution leads to 
decisions on the shape of a program of resea*'Ch on teach>ng effectiver^ess. 



Cutting Into the Web: 
How should we partHlon the Act of Teaching? 

Teaching can be viewed as a generalized role-function — an organic 
activity whose variations we try to clarify and try to relate to learner activity 
-or It can be partitioned m terms of a variety of sub-rotes and dimensions 
of activity related to those roles 

Gage' has severely criticised treating teaching as an undifferentiated 
function and attributed the slowness of growth m reliable indicators Of 
teaching effectiveness to the lack of partitions m the definitions of teaching 
which have guided so many investigations. This has reduced the likelihood 
of identifying relationships between teaching skills and pupii behavior. For 
example, questioning behavior by teachers is often studied and related to 
pupil outcomes We have to question whether any given aspect of teaching 
behavior is hkely to be effective through all the roles the teacher plays, or 
Without considering the complex of behaviors m which the targeted one is 
embedded, it appears to me to be very unlikely that any smgle teaching skill 
or even a combination of a number of skills is likely to be related very con- 
sistently to any kind of learning outcome except withm a specifically defined 
role Certain teaching skills may function beautifully in certain kinds of 
teaching roles and be totally irrelevant to others. Thus, to use the undiffer- 
entiated act of teaching as a dependent variable of training studies or as an 
independent variable with respect to all classes of learning outcomes m 
correlational studies seems to me to be unwarranted. The thinking which 
underlies most such studies is simplistic The investigator tends to get into 
a frame of reference where he hopes to identify easily definable skills that 
are likely to be related to specific kinds of learning outcomes and to search 
for a simple cause and effect relationship among these. 

/ prefer to see teachmg a$ an act which cor\tributes to the credf/on ot 
various learr^mg enviror}ment$ which fulfill certair\ kir^ds of roles in the lives 
of studer^ts. roles which have attached to them different kinds of potential 
benefit for the learner, in other words, it appears to me that teaching is the 
creation of a v^Wfety of learm^ environments from which learners can bene- 
fit in a var ^ty of vi^s in anothj^r paper for example. I have described a 
school in which learners visit several kinds of centers, each one of which is 
characterized by an environment that can help him grow in particular kinds 
of ways In the idiosyncratic center he meets counselors who help him to 
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find learning activities which will help him to grow on his own terms In the 
skills center he meets with diagnosticians who help him understand his own 
competence m reading, mathematics, and some other • basic" areas and 
define With him programs of study to remedy his deficits and to help him 
obtain further sKilis In the academic inquiry center he works with people 
who help him try ideas from the disciplines to learn the modes of inquiry 
that Characterize several of them in the social ecology center he works 
with others in social action developing the skills of active citizenship and 
learning to inquire into social problems and clarify the value ssues of his 
society Each center tries to foster a particular kind of independence m the 
learner — to teach him how to learn in a certain fashion. 



Learning Center by Function In a Hypothetical School^ 



Center 


Idiosyncratic 


Basic Skills 


Academic 
inquiry 


Social 
Ecology 


Function 


Personal 

Growth 

Program 


Mastery 
Of Symbolic 
Skills 


Concepts and 
Methods of 
Disciplines 


Valuing - 
Social Action 


Role of 
Teaching 


Counseling 


Diagnostic - 
Prescrtptive 


Inductive 


Group 
Inquiry 



For our purposes here, the importance of this illustration is that each of 
these centers i', characterized by a particular kind of learning outcome for 
which certain kinds of learnmg environments or modes are likely to be more 
appropriate than others. The teacher, the most important agent m creating 
those learning environments, probably needs different skills to work in each 
center How much teaching skills would Overlap from center to center is an 
open question. 

In today's schools we tend to employ teachers as generahsts. asking 
them to play a variety of roles analogous to those m the hypothetical school. 
A self-contained classroom teacher* for example* is asked to teach skills 
and concepts from academic disciplines as well as to counsel his students 
and to lead them m social dialogue. Just as in the differentiated staffing 
plan of the hypothetical school, each of these roles (and we might mention 
othere) requires a range of unique competencies embodied in one teacher. 
Ultimately, a program of research to define and validate competencies 
needs to take mto accc^nt these role differences Particular kinds of ques- 
tioning skills, for example, are appropriate to each model of teaching or to 
the environment which is most likely to produce the outcomes associated 
with each role. Many skills which may have important utility in one role may 
scarcely be relevant to another 

I believe that our collective effort to conceive of and validate teacher 
competencies depends on our ability to develop common definitions of 
teaching which partition the act so that we can search within defined fields. 
I believe that this was the major importance of Gage s recent article on the 
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microcntet ion approach, where he suggests that the general acts of teach- 
ing are far too broad and ilUdeftned to pernfiit systematic programs of re* 
search; but that if we identify particular Hinds of functions of teaching such 
as lecturing and counseling, we create a more rational field within which to 
develop programs of inquiry 

For my own work, pending a dialogue among the group of researchers 
working in this area, i am searching within the roles of the teacher which 
are defined m Appendix B I am attempting to identify the models of teach- 
ing which logically seem most productive for those roles testing the models, 
and trying to locate the competencies that appear necessary to t>fmg those 
models into reality inside of those roles 

Th# Phllotophlcal Side: Th« Avoki«RC« of TtehnophHIa 

The definition of teaching and the selection of environments are more than 
technical processes— we cannot study teaching as if it was an act without 
moral significance. For example, any model of teaching (the specifications 
of an educational environment) represents a philosophical statement. That 
IS. it emerges from a view of man — how he learns, and what he should 
learn. Consequently, it is a symtx>l as well as an instrument in the Uie of 
the learner The meaning of that symbol is likely to be as important an out- 
come of instruction as is any substantive or attitudmal learning which is 
likely to derive from instruction. 

For example, the use of drill and practice strategies in military training 
symbolizes the discipline of the military and an obedience orientation It 
represents a view of man as one who should be disciplined to collective pur- 
pose, who should learn to play his roles with precision and should con* 
fine himself to roles defined by an authority which is above him and in 
control of most of his behavior. The enyironnent of Summerhill, by contrast, 
symbolizes the importance of the individuaPs psychosexual development 
and the necessity for him to define life on his own terms and to place his 
needs for personal integrity and fulfillment above those of the normative 
requirements of his society. He has to negotiate the demands of his own 
needs with those of his fellows, rather than have the means of that recon- 
ciliation thrust upon him. 

In the search for the mcdek which are appropriate for the various roles 
of Ihe It^acher, and indeed when we select those rotes in the first place, 
we need to consider the symbolic effects and implications of our choices. 
If we search only for the most efficient methods, in the sense that we select 
only those models that result in defining learning outcomes in relatively 
short periods of time* we are making a philosophical statement as well as 
choosing an effective avenue. To avoid the implications of the symbolic 
value of the environments we select would be to beg the question of the 
nature of our existence and its meaning. At this time when there is so great 
a concern with accountability and when accountability is translated into a 
search for efficiency-related competencies, we must not forget that we 
create philosophical meaning in all of our actions. Because of this, to fail 
to research the symbolic effects of a model of teaching tc not justified. I 
am suggesting explicitly that one of the necessary dimensions of our re- 



ERIC 



BEST corv Ammu 



search is to begtn to undorstand the learner *s interpretation of the sym* 
boiism inherent m our educational environments and the kind of philosophU 
cal views which he is tikeiy to derive by exposure to them. We will presently 
return to the technical side of this aspect of research. As an issue, I place 
It early because the meaning of the remainder of our work is contingent 
on our handling of the symbolic worth of the types of teaching we choose to 
study and advocate. 

Potential Pluralltm: Dms our Retawch Converg* 
Toward a Monoinhic School or Dooa it DIvorgo 
Toward Educational Pluraiitm? 

I will not pretend to be open mmded on this issue. The question is plains- 
Do we hunt for a single, all-purpose model of teaching, or do we try to find 
the range of competencies that are appropriate for an educationally divers^^ 
world? 

The encouragement of diversity among mankind requires diversity m the 
nurturing of the young A pluralism of educational environments is impor- 
tant for two reasons. One '^ese is because it exposes to the learner the 
operationalization of a variety of points of view about the nature of man 
and his development. It symbolizes pluralism and shows him what it is. 
Second, exposure to different environments is likely to piomote different 
kinds of learning outcomes and thus increase the learner's repertoire Of 
knowledge and skills. Our research should anticipate a much greater degree 
of pluralism than we see today in most of our schools, or certainly have 
seen in our past. Pluralism, it has to be noted, is different from eclecticism. 
I do not believe that we should search for a model of teaching which is a 
compendiuTi of the most attractive elements of a variety of models. We 
should seek rather, I believe, to develop models on their own terms, borrow* 
mg only where it is clearly appropriate. An^ rather than searching for the 
optim^Tl teaching strategy for all purposes, we should search for ways of 
creating the pluralism of environments that will lead to a pluralism of human 
kind. Our research Should be relevant to today's schools, but not confined 
by them 

Behaviorism and Conservatism 

A few years ago Michael Apple* wrote a concise and pungent essay in 
which he pointed out tha* a search for teaching competence which relied 
largely on the study of teachers working in today's schools would be bound 
to have a restraining effect on educational progress, to pu* *t mildly. 

I have a great love for teachers ano still see myself as one. Teachers 
labor, however, under ungodly conditions. Elementary teachers are best 
described, from a behavioral point of view, as able to teach any learner 
anything under any conditions (to achieve any objective with any learner 
using any teaching strategy). In response to their role-demands they 
develop an eclectic, elastic style that is anything but a repertoire of ultra- 
sharp competencies. If we have learned anything from studying them, it 
IS that the demands of their role have funneled them into a style much more 
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that of a bureaucrat of learning materials than a master of technique. 

The t«me-cnnfines of the subject specialists, straight jacketed by the 
45-minute Class period, general course syllabuses, and adopted textbooks, 
have homogeniied them also into the skills demanded by the four moves 
described m Amo Beiiack s Language of the Cla$sroom^ the skills of the 
recitation Most Models of Teaching do not fmd the customary role of the 
teacher or the skills of this role congenial to their execution 

If we finally succeed m identifying the competencies most useful for 
today's teachers, they will look more like the skills of a country post- 
master than those of the creative artist or behavioral technician. Our task 
is. rather, to find the skills that will increase the capacity of fomorroWs 
teachers to vary— to discover the competence necessary to produce learn* 
mg environments rarely spawned m the present. &o*caiied 'Real World" 
of the classroom. 

If we can trust our theoreticians and researchers at all, we presently 
have a store of models of teaching which require competence rarely seen 
m today's classroom (For one example: a student teacher and his students 
produce, m one hour of microteachmg practice with the Synectics Model of 
Teaching, more synthesis^level communications than 30 experienced 
teachers produce altogether m 300 teaching episodes.*^) 

The Matching Modals Concapt: 
Building tha Equation from tha Studant. 

In addition, learners respond differently to any model of teaching We 
should not expect any teaching skill, let ilone any model, to have the same 
effect on all students Without a differentiAi concept there will probably 
never be such a thing as a validated teaching skill. What we may have 
ultimately is an elaborated matching models concept (such as the one pro- 
posed by David Hunt), in which the validation of a teaching strategy is 
reported m terms of its differential effect on a variety of learners, in other 
words, a program of research to validate teaching competence needs to 
anticipate the differential effects of environments on learners and build 
that capability into their research designs Our empii cal experience in the 
subject areas should reinforce this concept. The vast amount of research 
in reading methodology, for example, has shown very small differences 
among competing methodologies, except when one considers the char- 
acteristics ol the learner In other words, competence in the teaching of 
reading has to be conceived of partly as the competence to select a mode! 
that IS most appropriate for the growth of the particular learner. 

We may even fmo th it the effectiveness ol any given strategy may "wear 
off" after a penod of lime. The learner as well as the teacher may need 
variety to keep stimulated, and as he matures, his vacillating preference 
tor environments may be very important to him. 

Tha Sattinga of Taaching: Tha Context of tha Rolaa* 

When one conce've^i of teaching as the exposure of a learner to an edu- 
cational environment, he immediateiy perceives that the agent is not the 
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only mporianl element in the creation of the environment. Setting thecomas 
extremely important to both the learner and the teachers. In recent years 
in our work at the Agnes Russell School, we have created a variety of 
learning centers whicK support different kinds of teacher activity. In the 
skills center, for example, the teacher works with the assistance of a variety 
of diagnostic devices and a range of systems which can be used by learners 
to remedy particular kmds of skill difficulties. This teacher in the skills center 
is facilitated m the role of skill builder by these support materials. In addi- 
tion, he IS affected by the concentration of his efforts, which has enabled 
him to train himself in specific role-related skills. He can use programmed 
instructional methods and other systematic teaching methods with a far 
greater accuracy and fluency than can some of the other teachers in some 
of the other settmgs. Similarly, the teacher in the Sciencing Center is 
supplied With a storehouse of equipment which students can use to replicate 
science experiments. The presence of that equipment, as well as the 
teacher's competence in using it. affects Strongly the kinds of environments 
that can be created within that Center. Students can generate experiments, 
organize their equipment, and have places where they can keep the 
material for long periods of time while they're conducting a series of 
investigations 

The Film Center was supported with videotape equipment, motion picture 
cameras, still cameras, and a darkroom. The teacher thus supplied was 
able to Create an environment in which children learned the skills of filming 
and also could engage in production of films and television recordings. 

Not only were the physical settings important, but also the setting of the 
Centers in the community of the school. For example, the Film Center 
worked on a contract basis with small groups of children. The Skills Center, 
on the other hand, was responsible for the diagnosis of reading and arith- 
metic difficulties for every student in the school. Appointments at that 
Center were mandatory for all children The Sciencing Center negotiated 
with teams of teachers and their children to generate units of activity, as 
well as contracting with small groups of children. The possibilities of teach« 
mg are greatly affected by the fact that these Centers relate to their clients 
by appointment and by negotiation, as are the competencies needed to 
operate in the Centers. Competence is enormously related to both the 
material and social aspects of setting. The settings in many schools are so 
primitive that probably only a few kmds of educational environments can be 
created m them. The secondary teacher, working in 45-minute periods in a 
departmentalized organization has brief, if regular, contact with children. 
The roles he is able to generate m that kind of setting contrast greatly with 
those of the teachers m the Centers alluded to above. We need to increase 
knowledge about how setting influences competence, including how it is 
possible to activate different kinds of learning environments in a variety of 
kinds of settings 

Teaching as a Boost to Learning: 
Self-Dlrected Learnor Activity as tha Basatine 

When one conceives of teaching as the creation of learning environments, 
the emphasis moves toward the learner. If learning is seen as a product of 
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the learner s interaction with the environment, then it is primarily the 
learner s activity which produces the learning, A teacher as an agent can 
help shape the environment and modulate it to the requirements of the 
individual child, but in the final analysis, there is truth in the old clich^ 
that learning is done by the learner. Teaching can but facilitate learning, 
even m the most directive of the Modots of Teaching. Because of this, 
the effects of a teaching competence— the extent to which it contributes to 
learner growth— needs always to be measured against what the learner 
might do for himself. The re jIts of this can be somewhat surprising. David 
Hunt and I and our associate^ have conducted a series of studies in Toronto 
in which Marsha Weil and l have created environments styled after several 
models of teaching.' We have exposed to those environments students of 
varying personality structure. We have compared the substantive learning 
(recall of information and the integration of subject matter) of students 
exposed to those environments with that of children who read the material 
for themselves and conducted their own analysis of it. Even after five or six 
hours of teaching we failed to raise recall scores above those of the "do^it- 
themselves** control group After as little as one hour, however, we affected 
the numbers and types of concepts the children formed — the integration of 
material. In other words, we as teachers and the environments created by 
the models of teaching have a much greater impact on the learner with 
respect to "higher order * outcomes. 

Studies of the effects of teaching should be measured against the effects 
of self-directed learner activity. (The same principle applies in teacher 
education; training strategies should be measured against teachers* self- 
training.) 

The Multidlmenslonaltty of Educational Environments 

Educational environments consist of a variety of cognitive and social 
tasks set in a social context and a support system of materials. The learner 
thus is exposed to a social system, some kind of sequence of social tasks, 
sets of cognitive tasks, learning materials* and the personalities of the 
Individuals in the environment. The study of the effects of instruction is the 
study of the results of being exposed to a many-sided environment. We can 
expect multidimensional environments to have several kinds of effects 
resulting from a combination of the substances into which one inquires, 
the cognitive and social tasks, and the social relations. 

An instructional model or model of teaching is a pattern for the design of 
such an environment. In practice, inquiry into the effects of models of 
teaching is complicated by the fact that so many instructional strategies 
do not explicitly attend to a number of the important dimensions of the 
environment that they will inevitably create. We have to flush them out by 
careful conceptualizing and by y. ipirical work if we are to know what we are 
really studying— what the educational environment really is. For axample. 
the introduction into a school of automated instructional systems with 
individual study carrels influences the social as well as the instructional 
dimensions of the environment, although normally only the instructional 
dimension has been attended to. We have found, for example, that many 
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students do not like to learn in such a setting for purely social reasons. 

The closer we get to the study of learning environments, the more differ- 
entiated are their elements in Treble 4 1 there appears a task analysis of 
some of the dimensions of a concept-learning model of teaching. 

In the second 'vOlumn are the cognitive tasks which are presented to the 
learners; m the third column are the involvement options the options that 
a teacher has w<th respect to the social system which will be created. In 
the first column on the left is an identification of the kinds of learner skills 
required to respond to the different tasks. This particular sequence of tasks 
can be earned out m a little over an hour with junior high school aged 
children It provides some idea of the complexity of the environment that is 
created *vithin a simple model used for purely instructional purposes. 

In order to validate any given teaching competence, we need to analyze 
the dimensions of the environment thoroughly and to Study them with 
respect to their effects on the learner. Only in highly controlled studies can 
we vary small aspects of the dimensions of the environments, but it is 
necessary to employ highly controlled studies— to conceptualize the en- 
vironment very clearly and to manipulate variables systematically— if we are 
to obtain any reasonable knowledge about the effects of teaching on 
learners. 

The Multiple Outcomes of Educational Environments 

The learner m a multidimensional environment always produces a variety 
of kinds of outcomes.® There are substantive outcomes, such as the recall 
of information and the development of concepts for integrating it; there are 
attitudinal outcomes, such as a liking for the material, the method, and the 
teacher. There is the learning of social and cognitive skills. All of these need 
to be taken into account Specific educational models may promote par- 
ticular kinds of learning (what Marsha Weil calls model-relevant outcomes), 
but many models are likely to share educational outcomes, and may boost 
the same outcomes about equally. We need to test hypotheses about the 
existence of model-relevant outcomes and about cross-model effects. It 
is more likely that particular high-order and affective outcomes will be 
boosted by specific models than that paired-associate learning will. Ex- 
posure to material is Itkely to be such a powerful determinant of sheer 
paired associate learning by a highly-motivated learner that it is unlikely 
that very many models will stand out. The integration of information, how- 
ever, and the ability to apply information to new situations may vary much 
more. 

I believe that the transfer of learning she jld receive the greatest empha- 
sis in the evaluation of instructional strategies and hence, teaching com- 
petences should be validated much more by the efff:ct on transfer than by 
substantive outcomes. Consider, for example, the training of teachers. 
When we try to evaluate the effectiveness of strategies for training teachers. 
all of the important criteria are ultimately bound up in problems of transfer. 
If we teach an instructional strategy to a teacher, our only way of evaluating 
his performance is to learn what effects it has on children as the teacher 
transfers it to new situations. 
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It IS equally important to know what children transfer from any given 
instructional encounter. How do they carry the information to a new situa- 
tion: how do they use the concepts they have learned; how do they apply 
the principles that are t>eing developed: how do the attitudes they have 
formed "^nd the social relations they have developed affect their subsequent 
behavio. Simply to evaluate the performance of a model of teaching, or a 
given teacher who uses it in terms of what is obtained from the encounter 
itself IS to concentrate on the trivial and ignore the important. Probably 
the most important learning outcomes of models of teaching are those 
which are embedded in the patterns of the models themselves and affect 
responsiveness to future situations. For example, one outcome of a coun«- 
seimg environment is to equip the learner to respond to nondirective 
environments. In the case of teaming strategies centered on the modes of 
inquiry of the disciplines, the most important outcomes are those which 
have to do with his application of those modes to new situations. The model 
of learning embedded in the model of teaching is the critical transferable 
content. In the studies Dave Hunt and I have done recently, we have 
studied the process by which children acquire a model of teaching to the 
point where they can actively carry it out by themselves. We have exposed 
learners with different personality structures to training programs until they 
are able to behave m the mode of the model independently, without the 
assistance of the teacher. We have been very much surprised by the 
relative quickness with which learners pick up the hueristics of a model. 
What has not surprised us so much« but which complicates our lives 
consici'^rably, is the difference in transfer by children of different personality 
types. The more rigid children, for example, tend to apply a model more or 
less as the teacher uses it. following it almost as a formula, where the 
more complex children tend to transform a model and apply its heuristics 
rather than the formula of patterns that they have been exposed to. 

The study of transfer is enor^-nously complicated by the fact that the 
learner transforms what he learns as he applies it in new situations. This 
plagues the study of transfer in teacher education. A teaching skill which 
stands out clearly in a specific type of microteaching laboratory exercise 
may appear in quite a different form and even be almost burled in the events 
of the classroom. For example, several of my associates and I for several 
years studied and trained teachers' sensitivity to learners in simulated 
teaching situations. In the laboratory settings the teacher was given a 
teaching task and presented with one or more learners (actually role play- 
ers). The role players emitted cues which represented a point of view which 
conflicted with the teaching task. Until the teacher read the cues, attained 
the concept of the learner, and modified his behavior to build a bridge to the 
teaching task, he was unsuccessful in the exercise. We found striking 
individual differences in teachers* ability to *'read** the learner and modulate 
to him. and in the styles which were employed to accomplish this modula- 
tion. We also found great individual differences in teachers* ability to 
acquire the **skiHs** of sensitivity. When we followed teachers into the 
classroom, however, we found no reliable way to detect those skills. In 
the classroom the teacher was trying to read and modulate to many chil- 
dren, sometimes simultaneously. He transformed his skills enormously 
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in order to relate to the larger group of children. Sometimes we clearly 
saw versions of the behavior we were after; but in general we were unable 
to develop an economical and fair system with which to record it, let alone 
to judge its effects on children. 

Proposed Focus on Learning Outcomes, Presently, my associates and 
I are exploring the following types of learning outcomes as we explore the 
effects of different models. 

The substance of the transactions. We explore the acquisition of infor- 
mation (paired associate learning), concepts, and principles that integrate 
the material which is explored by the learner during any given period of 
time. Multiple-choice tests are used at the paired-associate learning level 
and essays or interviews are used to collect data which is content-analyzed 
to determine the kinds of concepts and themes which the learner develops 
in the course of his inquiry. 

Model-relevant concepts or skills. We have been exploring the develop- 
ment of tests to identify the particular skills and concepts which are generic 
to particular kinds of models. The synnoetics model of teaching, for 
example, attempts to teach children to think metaphorically. If they explore 
a given subject matter in metaphortc mode, they will acquire information 
and build concepts, but they also learn the concept of metaphor and how 
to use it. This IS a model-relevant concept, and the ability to use it is a 
model-relevant skill. Group investigation requires learners to work together 
to define objectives and to analyze one another's perceptions in a social 
situation. The ability to work with others to identify objectives becomes, 
therefore, a model-relevant skill for group investigation. Concept learning 
teaches children to enumerate, classify, and label data. The concept of 
classification, the concept of labeling, and the ability to classify and label 
are therefore model-relevant outcomes for that model. 

Transfer of substance of outcomes. This outcome involves the ability 
of the learner to apply what he learns to a new situation. If, for example, 
we teach children how to analyze a culture we would then investigate their 
ability to analyze a second one. or perhaps compare the second analysis 
with the first. 

Acquisition of the model of learning. I referred to this earlier when I 
described our attempt to determine what it takes to help a student learn 
to use the cognitive and social skills of a model of teaching to guide his own 
inquiry. A clear example is when we use inductive methods with children 
and then study their ability to carry out inductive inquiry independent of us. 
Situational tasks are most appropriate for investigation of this kind of Out- 
come. 

Attitudes toward salt inventories can be used to investigate how the 
learner views himself when interacting with any environment. Inventories 
can also be used to study the following affective outcomes: 

Attitudes toward material, (How does the student like the substance and 
learning material he encounters in an environment?) 

Attitudes toward the method. (How does he feel about the methods of 
teaching he experiences?) 

Attitudes toward the teacher (How does he feel about the training agent?) 

These are depicted in Table Two. 
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TABLE 4 2 

Type of Ou tcome Type of Measuring Device 

Recail of Matenai Tests - True - False 

and Multiple-Choice Items 



Integration of Material 

Transfer of Concepts 

Learned and Social and Cognitive 
Skills of the Model 

Attitudes about 
self 

teacher 
material 
method 



Content Analyses of Descriptions 
Given by the Learner 

Situational Tasks 
Inventories 



Tha Gttnaraily*Effectiv0 Teaching Skill 

What is the likelihood that we will identify a half*dozen skills which have 
powerful effects across models that will increase learning in many kinds of 
environments? 

As we make a task analysis of particular models of teaching, we find 
varying degrees of overlap. There are some models which appear to have 
almost no overlap with others. The skills of some fairly simple models seem 
to be entirely subsumed by fairly complex models. It is difficult, however, 
to identify any given skill except in the matrix of the pattern of a model. 
For instance, negotiating with a learner in a counseling situation is rather 
different than negotiating with the learner in a group dynamics model while 
the group is trying to analyze a situation ano develop hypotheses about it. 
There is negotiation involved in both cases and. at a microanalytic level, 
we can detect similarities. However, a teacher brings that skill into play 
differently within the different patterns. Marsha Weil conducted an inves- 
tigation which bears on this issue. ^ After an analysis of the interaction 
analysis patterns of a group of teachers who had learned several models of 
teaching* she extrapolated what appeared to be three general and widely 
useful skills that were related to difficulties in implementing the models. 
Each of these, moreover, was a fairly complex skill. For example, the skill 
of structuring consisted of the ability to implement several types of social 
relations in a classroom from a condition in which the teacher directed all 
of the activities to a mode m which the activities were negotiated with the 
children and to one in which the learners negotiated the activities them- 
selves. The skills, in other words* are not simple skills such as the ability to 
ask a particular kind of question or probe in a particular way. Weil built 
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instructional systems which could t>e used to train teachers to use those 
skills and did so. rating the degrees of performance that resulted. The 
teachers then learned several models of teaching and she correlated the 
degree of skill with the ability to acquire the models. Sha found that the 
skills, rather than having general effects as she had supposed they might, 
were modeUspecific, then were associated with specific models. We 
formed the generalization that the more classroom-unusual the behavior 
required by a model was. the more the skill that was relevant to that t>e- 
havlor was important to the acquisition of the model. In other words, a state 
of constructive structuring is relatively rare in most of today *s classrooms, 
and such structuring was extremely important in the group-investigation, 
but much less important in a model that utilized more or less conventional 
structuring. The skills, in other words, turned out to be model-specific. 

Eventually, we probably will find specific characteristics of teachers or 
types of skills that are quite general. To search for them at this time, 
however, before we have identified the skills most appropriate to particular 
models and determined how those degrees of skills relate to learner out- 
comes, seems premature. 



Duretion: Avoiding the Quick and Dirty 

How long does a learner need to be exposed to a particular environment 
before we can expect to see changes in his behavior? This question has 
critical importance for the design of Investigations to determine the 
effects of various models of teaching. To try to validate a competence in 
studies which are too brief for the effect of the competence to be detected 
IS wasteful It also tells us little if we employ studies so long that almost 
any reasonable approach has the kind of effect we are studying. There are, 
in the literature, an enormous numl>er of laboratory experiments in which 
(he duration of teaching was only 30 minutes or less. It is hard to imagine 
very many learning outcomes which would be seriously affected by inter- 
ventions that Short. 

It Should not be difficult, however, for us to engineer sets of studies 
designed specifically to develop principles of duration. At first these studies 
should probably be outcome-specific. For example, recall of information 
contained m written passages might be a dependent variable of experiments 
in which teachers interact with students for varying lengths of time. This 
type of experiment should be repeated for varieties of learning outcomes 
and approaches lo teaching. Without the principles which these studies 
would generate, we really have only ruler of thumb to decide how long 
experiments on teaching should be. If. as it turns out. transactions with the 
teacher should affect recall by the learner in a half-hour or an hour, we can 
productively conduct experiments in which the teaching episodes are of 
about that duration. If. however, it should turn out that one must teach for 
10 or 12 hours before the learner learns more than he does by reading the 
material by himself, then those experiments would have to be much longer 

It would also be important for us to track both the induced effects of 
teaching transactions (nurtured) as well as those that are directly taught. 
For example, one nurtures rather than instructs the development of self- 
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esteem through praise and support, but he can directly instruct the commu- 
tative principle It is reasonable to suppose that nurturant effects occur 
more slowly than instructional effects. Personally. I think this is why there 
have been so many unsatisfactory experiments in the field of reading. One 
does not simply instruct in phonics principles (direct effects) He provides 
a setting in which children can teach themselves a wide variety of things, 
including developing their vocabulary, forming concepts about coding and 
decoding, and developing the kinds of attitudes and reading habits which 
lead to real reading power. An experiment to evaluate a teaching strategy 
m the field of reading that lasts only 6 or 8 weeks, one may find very few of 
the important nurturant effects. 

A Systematic Program of Rataarch 
to Identify Teaching Competence 

I propose a six-^stage model for identifying and validating teacher com- 
petencies. The model is built directly on a resolution of the issues in ways 
which appeal to me 

Defining the Roles of the Teacher 

The process begins with the partitioning of the teacher s role. Ideally, 
the role is defined as a result of a construction of a model of the school (see 
Appendix A), so that we are looking for competencies for a characteristic 
kind of educational institution which deploys its staff deliberately and 
rationally. However, if we are faced with the problem of searching for 
competencies withm the present general teacher roles, then these need to 
be partitioned into subroies such as the teacher as individualizer of instruc- 
tion, the teacher as counselor, the teacher as instructor of groups, the 
teacher as lecturer, and so on. For purposes of illustration, let us work with 
three roles: the teacher as an individudlizer of instruction, the teacher as a 
small group instructional leader, and the teacher as a lecturer. 

The Specification of /?o/e Outcomes 

The definition of the roles should include a determination of the outcomes 
of the anticipated outcomes of the role, that is, how it is expected to function 
for the education of the student. For example, the role as individualizer 
might be associated with the teaching of basic skills in reading and mathe- 
matics at the elementary school level and the basic skills associated with 
any given subject matter at the secondary level. (For example, grammar 
in English, map skills in geography, and so on.) These role outcomes are 
extremely important because they will be the basis on which we ultimately 
wish to validate the conpetencie^ which are eventually identified. 

Developing Appropriate Settings and Support Systems 

The next task is to develop settings in which the role can legitimately be 
played and to specify the kinds of support systems that would be available 
to the teacher. For example, a teacher who is playing the role of an individ- 
ual of instruction and has available to him the resources of the I PI System is 
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«n a very different position than the teacher who has available to him a 
library of trade books in both cases the role can be played but the com- 
peiencies which are needed are likely to be very different. The I PI System 
provides the teacher with a diagnostic and prescriptive system that he can 
use on many of his tasks, whereas if he has open-ended resources, he has 
to determine the skills he will teach and perhaps to buHd his own diagnostic 
and tracking system. 

fdenWication of Approphate Modei$ of Teaching 

The next task is to identify which models of teaching enable the teacher 
to fulfill the role. There are two ways of doing this. One is by studying edu» 
cational theories and operationalizmg models of teaching from them. 
Marsha Weil and i have made a survey of available models, and we reach 
into this storehouse and match the existing models with the roles of the 
teacher For instance, we are able to identify a number of interactive and 
information processing models that seem appropriate to the role of small 
group instructional leader Concept learning, group investigation, concept 
detainment, etc . are all appropriate to the role. 

The alternative to theoretical sources is the Study of practitioners. We 
can ask teacher to play the roles in our prototype instructional settings 
and to Identify the patterns of behavior they use. thus developing models 
originated by practitioners. This is the method preferred by Arno Bellack 
and his associates at the Center for the Study of Teaching at Teachers 
College i am sympathetic to the model, tor i am also a member of that 
Center, but in my own work I tend to use the former approach. Either way. 
a series of models is generated which are role^appropriate. in the next 
step, a behavioral analysis of the models is made by comingling two 
methods One method is to obtain behavioral examples of expert teachers 
trying the mode»^ in the prototype settings A group of experts familiarize 
themselves with the model and try it out. so that we can get some idea what 
the specific behaviors are that thoy use The second method is to make a 
careful conceptual task analyse of the model, thinking it through stage by 
stage, and identifying the cognitive and social tasks appropriate to the 
model 

From these two processes we get a behavioral representation of each 
model. In the case of some models, it may have a relatively fixed sequence 
of phases, whereas others are very flexible, if the conceptual task analysis 
IS used. It Should be foMowed uy the empirical representation of the model 
by several expert teachers, so that a range of skills appears as well as a 
normative pattern 

Th«i Laboratory and Training Straams 

From this point there are two sequences of investigation: laboratory and 
training streams One sequence is research on the model of teaching. It 
includes an analysis of the function of different teaching skills m the model 
and an analysis of the effects of the model on learners in the short and long 
run The second stream of work is the training stream, which focuses on the 
development and testing of systems to tram teachers to the model o* each- 
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Figure 4,1 
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mg and the study of the effects when a range of teachers use the model in 
field settings. The two streams of work need not l>e separated in practice 
although It IS often convenient to have them separated. Whether they are 
done together or correlated, they have conceptually different purposes, 
in the laboratory research stream, studies are made of the effects of the 
model in controlled settings with an attempt to identity the kinds of skills 
reeded by learners to respond to the model, the differential responses of 
learners, and the kinds of outcomes which are boosted by the model m 
comparison with others. Thus the outcomes of the model of research are 
model-relevant o*iicomes. a differential model or model for applying the 
model with children of different characteristics, an analysis of the side 
effects of the model, and identification of the model of learning, in the 
training stream the results of the empirical and conceptual task analysis 
of the model are used to develop training systems to expose both teacher 
candidates and experienced teachers to the model. Teachers study the 
model, and they are studies m microteaching and field situations as they 
try them out An analysis o* their behavior leads to a more precise identic 
ftcatton of the skills which they have difficulty learning and which are essen- 
tial to carrying out the model. The results of this research lead to a rede- 
velopment of the training system. Another trial with the training sys^^ms 
determines the effect of the additional skill training and the effects of the 
model on learners To compare the results of the behavior of different 
teachers, controls have to be placed on the content which is taught, or it 
becomes impossible to compare outcomes. This phase of the investigation* 
however, represents the field validation studies. It may well be that, in the 
field, outcomes are legitimately different than those m the laboratory set* 
tingq of the model of research stream, but hopefully there is relatively great 
congruence between them. A comparison of the patterns of teaching in the 
field validation studies should indicate whether certain teachers possess 
skills which result m greater learning than do other skills, whether there are 
certain microskills which lead to greater effectiveness than any given model. 
As this program of work Is repeated with several models, one edges toward 
a position where he can determine whether there are any cross-model 
teaching skills of importance. 
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Task Force on 
Relating Teacher 
Competence to 
Pupil Outcomes 



Donald M* Modley 
DavM KrathwoM 

A substantial portion ot the time allotted to the task force was spent m 
small groups whose mandate was to formulate questions for discussion 
when the ent^e task force reassembled. Since not all of these questions 
were deaU with by the larger group, a summary is appended to this synthe* 

st<: 

The issue that seemed to receive t^^e t ^Mention had to do with whether 
Jt IS defensible to attempt to define * •< : ea.*^ compef'ince in terms 

of pupil outcomes The wording of •h.? * j ^ . /^r to the task force— to 
deal witM ihe problem of relating teach« co r o»i. i "^^j to pupil outcomes — 
rmphes that teacher competence «: i.ditent entity from pupil out* 

comes, and (2) that the relationship * * ■ i is an empirical ques* 
tion. While the point was disputt>d. r^. ■ g group concurred in the 
opinion that the teacher should be held ac oiintuc - tor competence rather 
than outcomes 

An analogy often cited was the fi edical one: it was pointed out that when 
a patient dies, the physician is not i eld respon'^ible so long as he has cor* 
roctly used the best known methods Kr diagnosing and treating the patient. 
It IS generally realized that the state of meu.'^al research is not sufficiently 
advanced that sure cures exist for many diseases Failures tend to be 
blamed on incomplete research data rather than on the incompetence of 
the individual physician. When the practitioner's competence ts called into 
question, his defense is based on his knowledge of and ability to apply the 
best known methods, and not on outcomes. 

Some question was raised about the fairness of the evaluation procedures 
jsed >n the program describe^ by Del Schalock. in which students are re* 
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Quired to pro^e their .Ib'iity to produce pupil learning before they may be 
certified as competent As one member phrased it. if the student uses the 
correct procedure according to his supervisor, and the pupils fail to learn - • 
who Should be held accountable for the failure- the student or the super- 
visor Or to paraphrase another comment is it ethical to demand that a 
teacner demonstrate more knowledge of the teachir^g process than those 
who purport to tDe trammg him possess ^ 

The state of research m teacher effectiveness • that is. research relating 
teacher behavior ^process) to pupii outcomes ^product) - is far less ad- 
vanced, everyone agreed — thar* research m medical science— that is. 
research relating the behavior of physicians to patient recovery. 

The defense of the Schalock program was that the learning outcomes to 
be produced were determmed by negotiation between teacher and super- 
visor, and were restricted to what would t)e reasonable to achieve with the 
group of pupils m question This is an irr.portant point, and goes a long w .y 
toward removing the objections cited above by ensuring (to a degreei that 
the student is not confronted with an 'jnreasonable task 

Serious questions were raised about the validity and reliability of such a 
procedure The reliability is. of course, unknown, and inaccessible to study 
Since each performance is based on different criteria as well as on different 
pupils, content, etc . further, not only the evaluation but the definition of 
the criteria are highly subjective Decisions as important as whether or not 
a student is to be admitted to the practne of a profession should be bas«d 
on measures ot high obiectivity and reliability, as well as validity. 

Questions ri^garding validity took two forms. The desirability of certifying 
teachers or the basis of ability to produce only short-term gams m limited 
content areas and with only one type of class was debated. The fact that the 
student is required only to produce what his supervisor considers a reason- 
able amount c* learninij was seen as tending to stifle progress by requiring 
the new teacher to be no more effective than the old. 

All of this discussion seemed to serve the purpose of sharpening a dis- 
tinction between the activities of research and evaluation. When we are 
evaluating teachers individually, we need much greater reliability (and 
practicality) than we need when we are doing research in teacher effective- 
ness 

The objective of research is to identify teacner competencies that are 
related to pupil outcomes m a probabilistic sense, that is. teacher behaviors 
which are most likeiy to produce pupil learning. Research studies are sta- 
tistical in nature and riea\ with groups of teachers; if we can show, for 
example, that the mean gam m a group which behaves m one way ir, greater 
than the mean gam m a group which behaves in another way. we have 
evidence of the greater effectiveness of one way of behaving. Reliable mean 
differences can (fortunately for us) be demonstrated with measures whose 
reliability for discnmmating between individuals is low. 

Measures of pupil outcomes are highly unstable; the prmciple source of 
errors of measurement are differences between pupils used one time and 
those used another time witt. the same teacher, if the average correlation 
oetween gams measures based on different groups taught by the same 
teacher is around 3 (as Rosenshi.ne reports), then it would be necessary to 
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average qams tu seviMi (jroups to get a measure whose rehaOiiily is 7S. 
Assuming that we are mterested m gams achieved over a one- week interval, 
this means it takes some seven weeks to measure teacher effectiveness. 
If we are interested m gams over one year, it takes seven years! The im- 
practicality of usmg mean pupil gams as a basis for teacher evaiuation— 
unless one is wiiimg to use very short term gams indeed is obvious And 
yet just such measures can be used m research studies to identify compe- 
tencies the possession of which wilt increase a teacher s effectiveness. 

If a teacher is evaluated according to the competencies he possesses, 
adequate reliability can be achieved with a reasonable expenditure of time 
and effort And when research begins to identify new ways of being effec* 
tive. we can begin to certify teachers who are more competent — and more 
effective- than the ones we certify today. 

When planning a study m this area, it is important to design a study with 
sufficient power to detect relationships that are none too robust: m par- 
ticular, sample sizes of at least 50 to lOO teachers are needed. Data should 
be collected that provide mformation. not only on the teacher behaviors 
and the pupil outcomes which are the primary concern, but also about 
variables likely to interact with them. Pupil characteristics (especially 
socioeconomic status and related characteristics) and type of curriculum 
seem particularly important It is dangerous to assume linearity. Recent 
research strongly indicates that m many cases, the regression of pupil 
growth on tearher behavior is curvilinear with optimum effectiveness 
achieved by teachers whose behavior is somewhere in the middle of the 
observed range of behavior. Outcome measures should reflect more than 
one type uf pupil learning, since it is likely that different patterns of teacher 
behavior will be most effective in facilitating different kinds of learning. 

To sum up. the general attitude of the group was positive— the consensus 
was that research in teacher effectiveness can increase our ability to tram 
competent teachers m performance- based programs by identifying the 
nature of teacher competence How rapidly these programs improve De- 
pends to a great extent on how extensive a research program we are able 
to mount and sustain 



Relating Teacher Competence 
to Pupil Outcomes Task Force 

A An area of concern to almost all members of the group had to do with 
the specification of the outcomes to which competencies are to be 
related Questions raised mclude the following: 

1 What philosophical base should be used m determining the desired 
outcomes'^ 

2 How may parents (and other community groups) gam access to 
the establishment of desired pupH outcomes? 
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3. How will political conslramls and decisions based on them— which 
may or may not be compatible with decisions about desired out- 
comes—be dealt with? 

4. Are there some competencies which relate positively to some out- 
comes and negatively to others'' 

5. Should long-term outcomes be used; and if so. for what purposes? 

6. How can the public be educated to expand its concept of outcomes 
to include affective dimensions? 



B. A second area of concern had to do with interactions between teacher 
and learner characteristics (and other extraneous variables) and 
their effect on relationships between competence and outcomes. 

1. What variables need to be taken into account in identifying pupil 
learning styles, teacher learning styles, etc.? 

2. Does the interaction between these two vary for such types of 
outcomes as self-concept, student attitudes, and mastery of cog- 
nitive skills'' 

3. Are there any generic teaching skills whose effects are invariant 
across such variables as these? 



C. A number of researchable questions were raised, including the follow- 
ing: 

1. What kinds of teacher competencies produce what kinds of out- 
comes With what types of pupils? 

2. How are these relationships affected by entering characteristics 
of pupils and curriculum variables, and how do they interact with 
teacher behaviors? 

3. is it possible to isolate the effects of teacher behavior from other 
environmental factors? 

4. What are some of these other variables? 
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5 is there enough known correlation between competenctes and 
outcomes to justify certification on the basis of competencies? 

6. Which kinds of outcomes are most reliably measured at the pre- 
service level? at the mservice level? 

7. Are there competencies which once acquired are never lost? Are 
there competencies which need continual reinforcement and prac- 
tice to be maintained? 



D. Miscellaneous Questions 

1 How can effects of setting and teacher role be separated? 

2. What variables are involved in research relating teaching and 
learning styles? 

3. Should the emphasis be on relating outcomes to competencies as 
such, or m identifying effective teachers and determining what 
competencies they possess? 



Appendix 

Summary of Questions Raised by Task Force A 

1 Using the regression model, couldn't you regress the teacher behaviors 
on performance of the outliers? 

2. How to overcome the resistance to judge certification eligibility based 
on pupil learning— uncontrollable conditions, differences in learners, 
home background 

3. The ' invasion of privacy" issue which will emerge when we use data of 
practicing teachers: effect on career, certification, salary, unless all in 
population are included. 

4. Are there ways of relating change in pupii performance to causal fac- 
tors without experimental designs? 

5. What about styles of teaching vs. child learning styles? 

6. What are the act-categories under consideration? 

7. What concepts of curncular structure and instructional structure are 
being assumed? 
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IS the teacher assumed to be performing in his individual acts under 
certain credible principles and ethical norms? 
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6 A Practical 
Approach to a 
Complex Problem 



Part 1 

Gtorge Gustafson 

A teacher training program must be considered from three points of view: 
program deveiopment, program research, and program accreditation. Cali- 
fornia IS unique among the states in that it has legislation which is both 
specific and. at the same time, provides opportunities for modifications in 
the three areas of development, research, and accreditation. One of the 
initiating elements for this legislation, the research and development of 
competency based teacher education, allowed us to move in some very 
constructive directions in a state where there are approximately 75 teacher 
training institutions. These institutions range from the very small to some of 
the major universities— the University of California, Stanford, and others. 

The teacher certification legislation in California established the California 
Commission tor Teacher Preparation and Licensing which is responsible 
for the certification of teachers. What the Commission tried to do was to 
design a program cycle that would allow a whole new series of events to 
occur in terms of development of professional education activities. This 
program cycle contains four pieces in a Simple paradigm. (Figure 6.1). 

The first of these pieces is planning, during which the institution o.oerates 
from a set of guidelines developed for* and adopted by. the Commission. 
These {guidelines provide direction to the program planning and development 
process. During this Stage the staff of the Commission works with the in- 
stitutions to develop a plan for teacher education. The institutions then 
submit the plans to the Commission tor approval, for conditional approval. 
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or for modification, based on the program planning guidelines. Program 
planning may take as long as two to three years. 

The California legislation has a very interesting and thoughtful part to »t» 
the requtrement tor involvement by persons outside the traditional teacher 
education program This requirement is incorporated into two aspects of the 
guidelines for program development by teacher education institutions. The 
first guideline requirement is that the world of practice must be involved m 
the teacher education program. The teachers, school districts, and the 
community must be involved in the planning process. The people in the field 
have some impact upon the development of the teache** education program 
by providing the institution with information about the types of teachers 
needed m the schools and the training requirements for such teachers. 

The second guideline requirement, which i think is equally important to 
the university and college people, is that teacher education must be a total 
campus commitment. The teacher education program may not be developed 
in isolation but must result from a coordinated, campus-wide, participation 
in the developmental process. 

The planning stage then moves into the second stage, implementation, 
during which the Commission staff works with the institutions in imple- 
menting and modifying the approved plans As modifications become neces- 
sary during this implementation stage, such modifications are in turn planned 
and implemented. 

The third stage m the process is that the institution does in fact have an 
operating teacher education program, based on the plan and the modifica- 
tions made during implementation of the plan. In the traditional sense, we 
said that the program begins with a series of ideas, plans, and operational 
notions, which, as they begin to be implemented, cause courses which are 
pref^ently being offered to be changed and moved into new directions. 

The real trick is to be sure that the programs which are developed, im- 
plemented. and operating at the universities have some relationship to the 
field. To insure the relationship of the programs to the field of public school 
teaching and to provide for program evaluation, we built In a system of 
collecting data which involves several steps— the collection of data about 
the person, the process, and the product 
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DATA COLLECTION SYSTEM 



person 

process] 

product 



Figure 6.2 

The first element to be considered in developing a program for teacher 
education deals with the person, so you coltect data and information about 
the people with whom the program of teacher education will be working. 

Then you do an analysis of the process. The colleges and universities 
begin to do things to the persons who are in their program. These include 
some good and some bad. but they are doing these to the students in vary- 
ing degrees. We have had some interesting battles over the process, and 
over what the institutions plan to do with the students in their programs; 
student teaching is one example. Everyone talks about competencies and 
the need to define the competencies which students must master, but the 
battle over student teaching centered on whether or not student teaching 
must consist of 15 units. The law clearly states that the requirement for stu- 
dent teaching shall be one semester of full-time student teaching or its 
equivalent. But the law does not define the equivalency, and the battle raged 
over whether it should be 18 quarter units or 12 semester units or what. 
There was little concern for the planning for the activities in which students 
would t)e involved during the student teaching experience. 

The third stage is the product which is the outcome of the person going 
through the process. One requirement of the program development guide- 
lines is that the institutions must state what their students will be able to do 
when they complete the program, the exit level skills or competencies. 
There is no attempt at this time« and hopefully there won't be for a long« 
long time, to specify the competencies which students must attain when they 
leave a program. There is no hidden list that specifies that these are the 25 
or 30 important competencies. The Commission is really working from the 
commitment that the institution must identify the types of competencies or 
skills which they presume to be useful and viable for teachers. The person 
who enters the teacher education program and the process of the teacher 
education program combine to produce an individual who will teacn in a 
school in California. 

The Commission will be concerned with collecting data to provide in- 
formation about how graduates of teacher education programs are func'* 
tioning within the public schools of the state. Thus, by looking at the person 
entering the program, the process of teacher education, the exit skills of the 
person leaving the program, and the ability of the teacher to teach within 
the public schools, the pieces fit. 

We have a system which has some inherent possibilities for flexibility. 
Ultimately, the Commission will have to look at the success of the process 
of teacher education as described by the institutional plans and as imple- 
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merited withm the teacner oducation »nstitut»ons They will have to look at 
the person, the process, and the product, m terms of how well the graduates 
deal With the real world of teaching m public schools and how well they can 
teach The information oPtamed Py looking at thf> product, the teacher m 
the public schools, will provide for a recycling of the teacher education 
program 

The need. then, is to design a system, a manner of collecting information. 
Which will put all of the pieces together, which will look at the characteris- 
tics of the entering teacher education students, the process of teacher edu- 
cation, the skills or competencies of the graduating teacher education stu- 
dents, and the success of the teacher m the pubhc schools 

As one step to accomplish this enormous task of systematizing the data 
collection process, i have designed a matrix. (Figure 6 3) There are seven 
categories m which we collect data about what is occurring withm the teach- 
ing process The first is one that is fairly traditional, that teachers are in- 
dividuals and as individuals, they have certain kinds of knowledge, whether 
It IS academic knowledge, public school required knowledge, or knowledge 
of the anticipated student learning m terms of the instructional Objectives of 
the school or district Teachers must have a knowledge base from which to 
organize their teaching both academic knowledge to convey to students 
and knowledge of the student learning outcomes which are important with 
the particular school d 'itrict 

A second category is knowledge about learning. If teachers are going to 
teach, they must have some knowledge about the learning process. They 
must know the ways m whiCh a teacher can assist with the learning process. 
They must know what stimulates and what depresses learning. 

The third category deals with student motivation If teachers are going to 
be effective, they must have some operational control over the factors 
whiCh affect student motivation to learn 

The fourth large area of concern is one about which there is a lot of talk 
today, diagnosis The talk centers on the need for professionals to be skilled 
at diagnosis, and the need for the development of tools to aid in the diag- 
nostic process Teachers need to identify the level at whiCh the student is 
functioning m a given subject or skill area. 

I think It IS almost impossible to deal with the business of diagnosis 
Without looking at the same time at prescription, category five. One of the 
most frustrating things for me as a professional, particularly as I consider 
the process of individualization of instruction, is finding that we have a large 
number ot diverse diagnostic procedures and that we end up with the same 
basic prescription the assignment of a page or series of pages m a text- 
book If you can t prescribe alternative strategies to meet the diagnosed 
needs of students, there is no point m domg all of the elaborate diagnosing. 

The next category. Six. deals with communication. Teachers must com- 
municate With a variety of audiences other teachers, students, the ad- 
ministration, and. very important m today s world, the community. So we 
must begin to collect information about the teacher as a communicator. 

The last category has to do with evaluation. Teachers need some knowl- 
edge of and some skills m evaluation of both individual student and total 
class progress Tnus. this matrix provides a means whereby we can divide 
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Pi0ur* 6.3 
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Student Progress 
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School Progress 
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into seven basic elements those teacher skills which institutions are telimg 
us are important teacher skills. Even though each institutional plan is 
unique, and the experiences through which students progress within the dif- 
ferent institutions on their way to becoming teachers are diverse, we have 
a consistent system for categorizing what institutions tell us are (1) the 
learning experiences of their students and (2) the exit skills of their gradu- 
ates The development of a system to categorize the stated experiences pro- 
vided within institutional programs of teacher education and the anticipated 
Skills of students upon completion of the programs was tough, given the 
uniqueness of each institutional plan, until we developed the matrix, which 
allows for classification of the process steps and the product definitions. 

When we look at students as they leave teacher training and enter the real 
world of teaching m public schools, we must recogn.ze what everyone 
knows, that districts have requirements and constraints which are unique 
to the individual districts When we t>egin to collect information about the 
skills of the teacher at the completion of £k teacher education program, then 
we can design a pre-employment training program at the district level to 
meet the specific, unique, needs of the district m terms of teacher com- 
potencies . 

The categories «n the first matrix define the broad areas of necessary 
teacher skills, but do not define any of the situations m which teachers 
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Plgur« 6.4 
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leacti To do that, wo n,iv*» Ot»siqne(1 .inother matriK As identified withm 
this matrix. iFigure 6 4| there are five Kinds of instructional settings *n which 
teachers are asked to function «n public schools While the matrm was de- 
signed with setf-contamed classrooms m mmd. it can also be used to 
examine open space learning environments and alternative school settings 
Whether the»e .«te walls ot not we are sl«ii dottimg with Tho five >nstriiCtionai 
settings 

The first IS the standard instructional setting where teachers instruct a 
large group of students, a small group of students, or work with students on 
a one-to-ons basis in this setting, teachers use a series of instructional 
methodologies and resources to meet the needs of the students withm the 
instructional setting 

The second setting is the library resource area In such an area, the stu- 
dent «s conducting research ith traditional research materials such as 
reference books, encyc'opedias. trade books, and programmed materials. 
Library reso'jrce areas are found m self-contained and m open space in- 
structional settings 

The third one is the n*»uiti- media center, distinguished from the library 
center by the presence of several modes of presentation of materials, such 
as movie ind film strip projectors While some multi-medm centers are 
stocked With the latest technological advances, ithers consist of less 
equipment and the sharing of equipment among teachers or schools Stu- 
dents may move to an area specified as the center and containing all of the 
t uipment. or the media materials may be brought mto the classroom 

The fourth settmg. one of the most important to me. is the laboratory 
Where the student has the opportunity to experiment directly, a place for 
children to develop skills m stating and testing ideas This is. agam. the 
type of activity which can be conducted anywhere and does not require a 
fancy location complete with the latest equipment. 

The last category deals with the community The community is any in- 
structional settmg outside of the school. The need to utilize the community 
as such a setting increases as we mvoive the commur.ity m defining the 
student needs These are the needs wnich the instructional program must 
meet m developing the goals for that prograiTi We cannot involve the com* 
munity in establishing the heeds and goals of an instructional program and 
then attempt to isolate the program from the community, or educate the stu- 
dents Without concern for an involvement m the community Agam. teachers 
in traditional school organizations, as well as teachers m open space nro- 
grams. can moke use of the community for a variety of instructional i A - 
ties 

These two matrices define the categories of teacher competencies and 
the instructional settings m which those competencies are employed Jointly, 
they allow for the classificat'On of the information obtained from the program 
plans from teacher training institutions, the data about program implemen- 
tation, and the information about the success of graduates when they are 
teaching withm the public schools This is the first step *n a systent to obtam 
and organize information about the totality of teacher education, looking at 
the person who enters the program, the process of teacher education, and 
the product, the teacher m the public school 
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Part 2 

Marjorift Powell 

leacher tr;<minq jnstjtutions ndmit siudents who meet entry cnteruT and 
ttien move tt\e students throuqM the procesb. their teacher education pro- 
qran^ designed to prepare students to teach m the pubhc schools The 
Canfomui Commission tor Teacher Preparat>on and Licensmg requires that 
•nstituttof^s describe t -^e program which they will implement The Commis- 
sion qrar^ts approval tor a penod of time to the program as described by 
the «nstitut«on and is establishing a procedure to insure that programs are 
implemented as described, or that moOitic%itions are documented 

While these steps mvoWe the person entering tt\e program and the pro- 
cess by Which the person is educated, ihey do not yet look at the product, 
the teacher teaching withm the school cla^r^room The important question 
IS still unanswered Will the students who move through the program as 
described ivui imp'emented by the institution be Sur.r.esstuI teachers when 
they have received their certificates from the Commission and have been 
hired to teacf\'' What skills are necessary to be a successful teacher*? 

In order to look for the answer to the question of what is a successful 
teacher, the Commission has undertaken a research study, the Beginning 
Teacher Evaluation Study, with funding from the National Institute of Edu- 
cation The purpose of the multi-year study is to identify teacher behaviors 
which are related to student achievement m reading and mathematics 

We have limited tne focus of the study to reading and mathematics be- 
cause of the necessity of confining the research to manageable proportions 
Such a study has a way o? expanding until the entire universe is included 
and the impact of the study is diffused to the point where nothing is learned. 
However, we do recognize Inat the elementary school curriculum contains 
more than the basic skills of reading and mathematics, and we hope that the 
work of thiG study will be expanded m the future to incorporate more areas 
of the elerrentary program 

The f rit year of the multi-year study. Phase i. was devoted to planning. 
After th£ development of an initial design for the study, the Commission 
sponso.ed a Research Design Conference m February. 1973 The confer- 
ence participants educational researchers teacher educators, school 
administrators, and sub|ect matter specialists, reviewed •he research design 
and made a series of recommendations Based on many of these recom- 
mendations, a design for the second year of the study was finalized. 

The second year. Phase ll of the study, focuses upon the development of 
instruments and the generation of research hypotheses Educational Testing 
Service (ETS) was selected to conduct the work of Phase II during the 
1973-1974 school year Experienced teachers m grades 2 and 5 form the 
sample for Phase ii. the first year of field work School districts were asked 
to participate, then experienced teachers were contacted, provided with 
information about the study, and asked to volunteer 

During the planning for Phase I', a variety of procedures was discussed 
for identifying the teachers to be included m the sample We considered a 
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process of nomination by alt of the teachers m the school or by the principal. 
However, even when the criteria to be used to make the nominations are 
clearly spec'fied. in many instances nominations are based on factors other 
than the specified criteria. We finally concluded that we wanted to study 
experienced teachers m the variety of instructional situations, without using 
nominations to identify teachers to be included in the study. 

The Phase n sample of 97 experienced teachers in eight districts included 
teachers in urban, suburban, and rural districts teaching students from low 
and middle income areas. Some teachers are in self-contained classrooms 
while others teach in open space programs. Some have mulU-grade groups 
of students, some are departmentalized, teaching only one or two subject 
areas. Several teams of teachers volunteered and are included m the work 
this year. The majority of the teachers have some form of teaching assis- 
tance—an aide, parent volunteers, or older students who tutor specific chil- 
dren. 

We are particularly pleased that teachers from so many instructional set* 
tings are involved m the sample during the first year of field work. We hopd 
to identify teacher behaviors which are important in a variety of instructional 
settings and must insure that the measurement instruments which we use to 
loentify and measure teacher behaviors are appropriate for, and applicable 
to. a variety of instructional settings. 

While the major focus of the study is on newly certificated teachers, a 
model must first be built, based on behaviors exhibited by. and related to 
student achievement in the classrooms of experienced teachers. For this 
reason, experienced teachers will be studied during Phase II and will be the 
major part of Ihe sample in the work next year. 

instrument developr ient during Phase II has included the development 
of student achieveme.it tests in reading and mathematics, specifically de- 
signed to measure student achievement in reading decoding, comprehen- 
sion, application of skills and attitude; mathematics computational skills, 
concepts, application of skills, and attitudes. 

Fall data collection was conducted during f^ovember. 1973. Student 
achievement will be measured again in the spring to provide a measure of 
student growth during the school year. Most of the student instruments have 
been developed specifically for this study, so normative information will not 
be available. 

Instruments have also been developed to measure teacher knowledge of 
the teaching of reading and mathematics, and teacher verbal ability, or the 
ability of the teacher to process information. The latter instrument included 
subtests of vocabulary, memory, classification skills, inductive and deduc- 
tive reasoning, and creativity. A film test is being developed to assess the 
skills of teachers to diagnose the reading problems of students and the 
range of alternatives which the teacher would prescribe to meet the iden- 
tified need(s) of the students. The test is being developed because we 
recognized the importance of the skills of diagnosis and prescription, but 
we were concerned that we might not observe enough instances of diagno- 
sis or prescription to adequately measure the skills of the teachers during 
the somewhat limited observation period in the classrooms. 

Observation procedures and two instruments have been developed by ETS 
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specifically tor t^»e nuilttplu cl.issroom observations withm this study The 
observations are occurring from February through April of the 1973-1974 
school year Differing numbers of observations are bemg done, using the 
two observation instruments The teachers indicated that thev would prefer 
as many observations as possible, since they did not believe that a few 
observations would provide enough information about important teacher 
behaviors The differing observation patterns will also allow tor the analysis 
of the reliability of the instruments and the observers, and a determination of 
the varyir^g types of information obtained through the use of each instrument 
when the same classroom is observed at the same time by observers using 
different instruments 

Teachers are also be<ng videotaped, so that further analysis of teacher 
behavior can be completed The teachers will have an opportunity to view 
their own videotapes after the taping is completed, providing the teachers 
with an opportunity to see themselves m instructional situations in addition, 
information about both teacher and student background has been collected 
A series of work dianes has been developed to collect information about the 
instructional program within the Classroom throughout the school year and 
the particular ir^struct'onal events during the specified weeks of the year 
The teachers work withm an instructional setting, with resources available 
to them, and with specific goals for students The work dianes are designed 
to obtain data concerning the instructional resources and goals. 

The school ciirrtate is being measured through questionnaires for teachers 
and principals and an interview with the principals All school instruction 
occurs Within the setting established by the persons involved m the school, 
the principal, the teachers, and the students This environment, or climate, 
has an impact upon the instructional situation and therefore must be as- 
sessed in order to adequately determine the relationship between teacher 
behaviors and student achievement 

The consistency of teacher behavior throughout the school year will be 
determined through the consistency substudy, which involves the videotap* 
ing of a sample of teachers three tnties during the school year, and the 
analysis of the videotapes in terms of the consistency of teacher behaviors 
across the three videotapes 

The stability of teacher performance, as measured by student achieve* 
ment. will be assessed through the collection ot mstoncal test data for 
students m the classrooms of the sample teachers m previous years. This 
task s especially difficult due to the differences m testing programs across 
the eight participating school districts. 

The dependeiit variable m the Beginning Teacher Evaluation Study is 
student performance, a measure of growth from fall to spring The mde* 
pendent variables are teacher performance (behaviors), teacher char* 
actenstics (years of experience, etc.). student characteristics (age, lan- 
guage background, etc ). and school characteristics (size, special pro- 
grams, etc ) 

Phase III will be designed to refine instruments and to test the research 
hypotheses which are developed in Phase li. to provide empirical data con- 
cerning the relationship between teacher behaviors and student achieve- 
ment in reading and mathematics During Phase ill, beginning and expen- 
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enced teachers will be mduded m the sample of teachers* to test tho hy- 
potheses With both types ot teachers 

Phase in IS planned to cover three years, with the first year devoted to 
additional development and field testing of instruments and research hy* 
potheses A series of special studies are planned for the first year. Ati of 
this work will lead into the second year of Phase ill. dunng which a large 
scale field study will provide a test of the research hypotheses. The third 
year will focus upon a review of the implications of the study for teacher 
certification and teacher preparation. 

To review these implications and to obtain information about training 
procedures, a senes of small studies will be conducted m which teachers 
Will be trained to utilize the behaviors identified during Phases II and III. 
These studies will provide both a test of the relationship between the 
teacher behaviors and student achievement, and information about one or 
more methods of preparing teachers to exhibit these behaviors. 

While I am confident that the study, over several years, will identify im- 
portant teacher behaviors and will provide information about procedures 
for training persons to implement the behaviors, I would be unrealistic if 
I expected that the study would result in a firm list of behaviors which could 
be defined as vital for all teachers m all teaching situations. Rather, i antic- 
ipate that we Will identify some behaviors which are important to all teach- 
ing Situations and other behaviors which are important to specific instruc- 
tional settings or the teaching of specific skills. 1 am convinced, further, that 
instirutions will want information about the one or more methods of teaching 
students to exhibit the identified skills 
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Classroom 
Observation 



Robtrt S. Sow 

A point of view which has been expressed with increasing frequency is 
the view that systematic observation of classroom behavior has not been 
a useful measurement procedure for identifying teacher behavior which is 
related to pupil growth. Rosdnshine* concludes that ratings are more ef- 
fective than observation of teacher behavior using specific behavioral items, 
and Heath and Nielsen evaluate work in this area and conclude. 

Last, given the welUdocumented strong association between 
student achievement and variables such as socio-economic 
status and race (see Coleman and others. 1966). the effects 
of techniques of teaching on achievement (as these variables 
are conventionally defined) are likely to be inherently trivial.^ 

Whether this extreme position is accepted or not. it appears that research 
which has attempted to relate measures of classroom behavior obtained 
from relatively specific behavioral items to pupil growth may have failed to 
produce as much as it could have because of a number of methodological 
difficulties. Since these apparent pitfalls in the past use of observational 
methods are likely to be related to the success of these methods in PBTE. 
they will be reviewed briefly here. 



Methodological Problomt In tho 
Past Um of $yttom«tlc Obtorvation 

There have been numbers of weaknesses in the studies which have 
employed systematic observation in the past, including the power of the 
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typical study, the nature of the observational measure, the nature of the 
achievement measure used as an outcome, the statistical analyses em- 
ployed, and others. 

Power A major difficulty with research using classroom observation 
has been the limited power of the typical study. For example, in the review 
cited above. Heath and Nieison divide their summary into studies with IS or 
more teachers and studies with fewer than 15 teachers, if 15 is taken as the 
minimum number for a study to be taken seriously, as their division of 
studies implies, and if the correlation between two variables as m09$ur9d 
is .30 in the population, then each study will have only 19 chances Out of 100 
of finding that result significant at the 5 percent level' But this is not the 
worst of the problem. Cohen argues persuasively that if the relation between 
two true m§asur0s is .30. a reasonable degree of fallibility in our measure- 
ment procedure would reduce the observed relation to .10 between the 
variables as measured, if that value is taken as a reasonable estimate of 
the correlations which we attempt to test, as Cohen argues, then only 6 out 
of 100 relations which are real would be found to be significant. There is. 
of course, the counterv^lllng influence that when large numbers of statis- 
tical tests are done, some will be significant by chance. Thus, one influence 
tends to produce too few significant results, and the other tends to produce 
too many. Since both of these influences are at work simultaneously, it 
would be surprising if there was very great consistency of results across 
studies m which the numbers of teachers are small. This, of course, sup- 
ports the conclusion that our knowledge base is weak; but it argues that the 
basic difficulty may not be the insensltlvity of the neasurement tool, but 
the fact that so many studies have been done which possess limited power 
(and that these studies are taken as seriously as studies with greater power). 

SimpUstic Statistical Analysis. Another major weakness of past class* 
room observation research Is probably the statistical analysis, which has 
usually stopped at the level of linear correlations. Very few analyses have 
tested the possibility that relationships might be nonlinear, and even fewer 
have examined interactions — the possibility that the simultaneous effect of 
two behavior measures might be different from the effect of each taken 
alone. Intuitively, both ideas seem appealing. It seems unlikely that a 
teacher behavior which is desirable in some amount is more desirable as 
the amount mcre.^ses. without limit, if some praise is good, is more praise 
always better? is more indirectness always better? is more drill always 
better? We have found strong nonlinear relations between a teacher 
behavior which controls or directs pupil behavior, and pupil growth, in four 
different sets of data.* Yet a linear correlation assumes a "more is better ^ 
relationship. 

It also seems piobabie that the meaning of any particular teacher be* 
hdvior is moderated or Influenced by other classroom behaviors. Simple, 
linear correlations also fall to recognize this possibility. For example, in 
our recent work, the extent to which the teacher was central in the work 
of the classroom was unrelated to any measure of pupil change.^ But if the 
teacher was often central and also often chose the problem to be worked 
on (not necessarily in the same activity), pupil growth was less than if the 
teacher did one or the other with some frequency A number of such inter* 
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•ictions agroiHi w»th iMt h ottw »uui with the nonlinear relationships m 
indicatmg that greater amounts of pupil growth were associated with inter» 
rtieduite amounts ot structure or control by the teacher 

There were also nurribers of •r.teractions between pupil characteristics 
and the dasbtoonn behav»or which was associated with most gam For 
example. nicreas«ng amounts of gentle teacher control (a very gentle. 
i<nobtrws<ve iioncoercive styles were more strongly associated w»th greater 
subject matter gam for disadvantaged pupils over the summer than for 
advantaged pupHs 

Empif icaiiy. the same study found that simple linear correlations produced 
about one and a half times as many significant results as would be enpected 
by chance, whereas an analysis of nonlinear and interaction terms produced 
between three and four times as many s»gnificant results as would be ex- 
pected by Chance So it seems likely that another problem m past research 
has been the mfrequency with which analyses have adequately dealt with 
the complexity of the relationships that exist between measures of classroom 
phenomena 

Complentty of Achtevement Measures. Still another problem which 
seems 'iKeiy to bear on the application of observation m PBTE has been the 
failure to specify the nature of an achievement measure beyond merely 
naming the subject matter represented Withm a subject matter, a measure 
may «nvoive learning which is Simple and concrete, or complex and abstract, 
requiring association or information processing For example, an arithmetic 
measure might reflect only knowledge of the multiplication table at the 
level of rote memory, or it might represent complex problem solving. It 
seems probable that rather 'lifferent styles of teacher behavior would be 
associated with most gam for eacn kind of learning Three separate studies 
have supported this idea with varying deyrees of completeness 

in the research which has been reviewed m this field, if measures differ* 
mg m complexity or abstractness were used as outcome measures, it would 
not be surprising that there is little consistency m the results ■ 

Complexity of Ciass^oom Behavior. Still another possible source of 
difficulty m past observational research may have been the failure to 6is- 
tmguish between subtly different classroom behaviors For instance, teacher 
behavior which was directed at confroHmg the behavior of pupils, can use- 
fully be distinguished from teacher behavior which was intended to control 
or limit pupil thought processes It is possible for the physical activity of 
pupils to be restricted, and yet for them to be engaged m complex problem 
solving or m very creative endeavors In fact, previous data indicated a 
positive relation between a freeing style of teacher *pupil interaction and 
pup«i growth tn creativity, but a negative relationship between degree of 
pupii physical movement and the same creativity growth measure.^ There 
IS no way of knowing the frequency with which this distinction is maintained 
at a theoretical level, but several sets of data indicate that teachers m 
classrooms rarely make this distinction 

in general, distinctions of this sort which have emerged from the use of 
classroom observation, seem likely to clarify the nature of the teacher's 
task and the recording task which represents it On the other nand. it seems 
possible that failure to recognize such distinctions may have led to some ot 
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the contusion m results which is present in current literature Not infre- 
quently, conflicting results are a consequence of the failure to recogniae 
additional important variables, and it seems possible that this may be the 
case here. 

Single versus Mu/f/p/e Cotf/ng. Another major problem which has not 
always been faced in the use of systematic observation has been recording 
specific behaviors m isolation. For example, the frequency with v^hich a 
teacher asks analytic questions may be a useful outcome measure to assess 
the consequences of a module intended to teach students to ask analytic 
questions, but perhaps the fact that the student-teacher asks analytic ques- 
tions will not make very great differences in the achievement gain of pupils, 
unless these questions are asked at aoproprlate points in a developmental 
sequence of learning experiences. For example, Taba and others concluded 
that a teacher would be unable to maintain a higher level of thinking in pupil 
inquiry if an insufficient amount of attention had been given to gathering 
the requisite facts before the higher-level thinking began.* 

Recent developments in multiple coding (for example, simultaneous.^ 
coding who speaks to whom, the level of thinking in the interchange, the 
affective tone, etc ) are important attempts to record such complex and 
sequential information and have recently been reviewed by Flanders. 
The beginning work was the matrix tabulating procedure of the Flanders 
System which captured, one step at a time, the sequence of occurrences 
m classrooms 

But there IS a possible problem in the nature of the items which are used 
in instruments intended to record multiple aspects of an event, or sequen- 
tial events. The problem is that the items which represent each aspect (who 
and to whom; whether direction, instruction, or question; the affective tone, 
etc.) must t>e small in number in order to make the coder's task manage- 
able This means, in turn, that they must be broad. The fact that a direction 
was given, for example, would be given in a single code, in contrast, in the 
Florida CUmatB and Control System, direction giving has 20 codes to scale 
the degree of coerctveness with which verbal directions are given, from very 
gentle, noncoercive ones to very harsh, coercive ones.** Correlstions of 
Items with pupil outcome measures in the lower grades suggest that gentle 
control items relate positively to growth, intermediate items do not relate, 
and harsh, coercive ones relate negatively. As a consequeitce. when all 
direction giving is assigned to one code, relations with outcome measures 
may be masked by the cancellation of positive and zero, or negative, rela- 
tionships. If. as we have come to believe, the kinds of observational items 
which are most likely to be effective in identifying classroom behaviors that 
Will relate to pupil growth will often to items which are relatively narrow, 
specific, and subtle rather than obvious, then there is likely to be a trade-off 
in loss of information by the use of items which can be used to multiple 
code events or sequences, in addition, it seems likely that the multiple 
coding of events or sequences will complicate the problems of analysis. 
Each combination l>ecomes a variable, so that combinations must be con* 
solidated in some way. and information may be 'ost. 

It seems clear that two approaches to dealing with the complexity of 
classroom phenomena and relationships are in use. One records events in 
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greater detail, but without a multiple record* then develops combinations 
and interactions by statistical analysis, the other records multiple aspects 
or sequences of events and then combines them ii seems dear that the 
nature of the data will differ, and which approach will be more functional* 
or more functional for what purpose, is an empirical question. Several in- 
struments of both sorts will be reviewed later 

RelmbfHty versc/s VaMtty Traditronal concerns over reliability may also 
lead the developer of observation instruments astray The items which are 
most easily observed reiiaNy are the relatively obvious, global, molar ones 
such as. "Is the teacher front and center in the teaching process?" While 
observers agree readily on how such an item should be tallied, the measure 
does not seem likely to be a useful one. indeed, there are some suggestions 
that the teacher behaviors which most often relate to pupil gam are She 
relatively subtle, fleeting ones, m contrast to the more global items for which 
reliability is ed(>er to obtain.'? Thus, it seems likely that if the developer of 
an observation instrument focuses first on the reliability of items as he is 
developing them, he may risk giving up potentially powerful items and 
retaining ones which have little utility. Ultimately, in applications in PBTE. 
the decmion should surely be influenced by the question of what is to be 
measured, but the temptation to give up items which are difficult to observe 
reliably may need to be resisted. 

Th# Plac* of 
Obt«rvallon In PBTE 

Observation may play at least two. and perhaps three, rotes m PBTE. 
First. It seems likely that the observed behavior of the student-teacher in 
training will often be used as a measure of the success of a module or a 
larger unit of the program. That is. it would be an outcome measure assess- 
ing the success of the student and of the tra ning material. Second, in the 
long run. it seems likely that observational measures will need to be em- 
ployed to test whether the differences in teacher behavior which the pro- 
gram produces, in turn, are related to differences in the achievem«^nt of 
des«rabie objectives by pupils That li. m the simplest terms, observation 
may help to answer two qua$tions: does the program make a difference 
m the behavior of its graduates? and. do these oehaviorai differences m the 
graduates make a difference in the pupils they teach? 

It seems wise to keep these two functions separate. Each alone will 
probably be a difficult attainment to assess, but to attempt to relate program 
differences to differences in pupil outcome seems almost to foredoom the 
validation attempt at the start. The slippage is great enough at each link 
in the process to make the demonstration of relationships difficult, but if 
both sets of linkages are included in the experimental test, the difficulties 
appear to be increased materially. 

Finally, observation systems may be used as a way of teaching desirable 
behaviors to student teachers m training. There is a growing body of litera- 
ture indicating that simply teaching a student a set of categories which 
describe behavior is surprisingly effective in modifying his behavior in a 
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Tht Uttlulnttt of Htorarehitt of lltmt 

As the faculty at our university worked through the question of whether 
they wished to commit themselves to working on a performance base, the 
ob)(«ctiori was sometimes raised. "I'm not interested tn behavior at the level 
of whether the student can thread the film through the projector * In order 
to provide a means tor talking about this problem. Figure 7 1 was produced. 
It should not be taken as a sm of standards to be taken seriously, but only 
as an example of a way in which specific t>ehaviors might be organized into 
successively more abstract and comprehensive conceptual schemec. The 
top row across the table represents a kind of broad hypothesis or belief 
about a desirable skill for a trainee to have or a desirable situation for him to 
create in a classroom, it would be accepted by most educatcKs. but because 
of its level of abstraction, would probably have different meanings to the 
different people who accepted it It is not. as it stands, measurable. If wo 
move down m a row in the figure, however, the concepts of Gentle Teacher 
Control and of a Warm Chmate are somewhat more distinct and behavioral, 
and yet are sufficiently comprehensive that most people attach values to 
them. If we move down to a still lower level in the table, a series of mea* 
sures is identified which makes up each of the desirable classes of class- 
room behavior. Each of these behaviors, in turn, is defined <n a manual 
which specifies with some detail the settings and the aspects of the be- 
havior which qualify for each of the measures cited, it is a kind of implicit 
'appendix" to the measurement scheme. Given such a system, the term 
Gentle Control or Positive Climate has relatively clear meaning, and there 
18 relatively little dmagreement about what the term means when it is used. 
At the same time, it represents behavior at a level of generality that appeals 
to many people as being of some consequence. 

In discussions with others, it appears to be fruitful and to increase under- 
standing and communication to clarify several levels of the hierarchy. If 
specific measurable behaviors are cited, it helps to give them moaning to 
answer such questions as. "Why would you care about that?*' "Why is it 
important?" On the other hand, when broader labels are used, greater spec- 
ificity IS sometimes achieved by asking questions like. "What would a 
teacher be doing if he were doing that? or "What would he be doing that 
would be different than if he weren't doing it? ' One advantage of such a 
procedure lies m avoiding the need to deal with hundreds of behavioral items 
for most purposes. Rather* a considerably smaller number of terms can be 
used which stand for the larger numbers of behavioral items that make 
up each complex. 

The critical requirement is to verify statistically that the items which 
make up a composite are positively interrelated if not. the unrelated or 
negatively related items may« at the least, muddy the concept, and at the 
worst, may destroy it operationally. The frequency with which an item 
empirically differs from others with which it seems logically to belong under- 
lines the importance of this empirical check. 
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Som« WM«ly Uttd Obttrvatton Inttnimtnto 

The first three of the systems which are described use large numbers of 
items, but do not code sequences or provide for multiple codes which can 
later be identified as describing a single event. The third is an extension of 
the Flanders System of interaction Analysis^ which was the first to begin 
to represent sequence, and the last three are more recent instruments which 
code sequence or multiple aspects of an event. All are ' general purpose/' 
in that they are not designed to assess a particular competency or skill or 
to describe teaching m a particular subject matter. Several are designed for. 
or have been mostly used in the lower grades 

TNt Flar((S0 Clim%t9 and Control Systant (FLACCSt, This is a modifica- 
tion of the South Carolina Observation Record/' which drew heavily on the 
Hostility* Affection Schedule^* and the earlier versions of the Observation 
Schedule and Record (Medley and Mitxei 1958, private communication). 

The first section is organized around the concept of direction and control 
of the classroom by the teacher and the response of pupils, including the 
teacher*s role and the organization of the csess. Twenty items of verbal 
teacher control are scaled into five levels of coerciveness ranging from 
"Guides. Suggests." to 'Orders. Commands** and smaller numbers of 
nonverbal items are similarly scaled. The response to teacher control is 
represented by pupil items, such as assumption of responsibility for class- 
room order, pupil task involvement, obeying, resisting or disobeying, and 
the amount of socializing. The second section, affect expression, records 
the eight combinations of teacher versus pupil, verbal versus nonverbal, 
and positive versus negative affect. 

Th0 UachBr Pracffces Observation Racord (TPORh This is an instru- 
ment developed to measure a teacher's practices in relation to John 
Dewey's Experimentalism. with each item describing a teacher behavior 
that is widely practiced in schools. 

It records whether the teacher or pupil is the center of attention, the 
extent to which pupils are active or passive, nnd the amount of freedom that 
pupils are permitted, it describes the nature of the problem as organized 
around the concerns of pupils or the concerns of the teacher or textbook, 
as well as identifying the difficulty of study topics. Subject matter is classic 
fied as to whether the pupils or the teacher assume primary responsibility 
for locating It, and whether it is taken from a textbook or a wide range of 
sources. Whether the teacher evaluates the pupils' work or the pupils en- 
gage in self-evaluation is recorded, along with the degree to which the 
classroom tasks are differentiated for individual pupils. 

Florida Tanonomy of Cognitive Behavior (Cog Tau). This instrument is 
based on The TaMonomy of Educational Obiectlves: The Cognitive Domain, 
as it was modified and extended by Sanders^' to assess teacher lesson 
plans and teaching materials. The Florida group converted Sanders* instru- 
ment to one for live observation in the classroom.^* The levels of the instru- 
ment are: 

1 . Memory - recognize or remember information; 

2. Translation - alter the form of the material— figurative to literal, be- 
havioral to verbal— but not to change or evaluate the ideas; 
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3 Interpretation - identify similarities or differences, relate supporting 
evidence to a generahiation, or carry out a specified operation; 

4. Application - bring together and use previously learned material such 
as deciding what mathematical operation is appropriate to solve a 
problem, 

5. Analysis • applying tiie rules of logic to the analysis of a problem, 
or inferring feelings or motives; 

6. Synthesis - bringing ideas together, but reorganizing or changing 
them in such a way as to produce something new; 

7. Evaluation - establishing a set of criteria, and then evaluating. 

The original instrument was intended for all grade levels, but since a 
performance which is routine and low level when well learned may be high 
level while being learned — reading, for example — a new version of the In- 
strument was developed for kindergarten or first grade. 

Although It IS sometimes believed that the higher levels are more appro^ 
priate for older pupils than for younger, experience indicates that this is not 
true. The difference, of course, is that simpler materials and concepts are 
involved at lower grade levels. The development of a Piagetian concept such 
as conservation would fall at the level of synthesis, and the discussion that 
accompanies a story or a reading lesson may deal with questions such as 
"What else might Jimmy have done?'* (synthesis), or "Would It have bean 
better if Jimmy had done something different? Why?" (evaluation). 

Reciprocal Category System (RCS), This is an extension of the Flanders 
System of interaction Analysis, which is one of the most widely used in- 
struments and whose well-known 10 x 10 matrix first permitted analysis 
of sequences. 

In the modification by Ober. Wood, and Roberts, the seven teacher 
categories were expanded to nme7' Teacher lecture is divided into that 
which Is responsive to pup'is. and that which is teacher initiated: the 
category of teacher criticism is divided into a category for correction without 
criticism, and one with criticism. Category 10 remains silence and confu- 
sion as before. Each of the categories is redefined, however, so that it can 
be used for pupil talk as well as for teacher talk. That is. teacher amplifica- 
tion of a pupirs idea is coded as a 3: a pupil amplification is a 13. Each 
category is changed from a teacher category to a pupil category by adding 
a "1* as the first digit The observer, then, learns nine categories as he did 
with the Flanders System but has 18 to work with and. as a consequence, 
the same variety of pupil talk is recorded as teacher talk. Practically 
speaking, this modification offers greater richness of data at little Increase 
in the complexity of the observer's task. A strength of the matrix procedure 
mitiated in Flanders work is the capt:«ring. one step at a time, ot the se- 
quence of occurrences by the way the categories are tabulated. An inter- 
esting feature of the RCS is that it can be broken down into four submatrices: 
teacher-teacher talk, teacher-pupil talk, pupil-teacher talk, and pupiUpupil 
talk. 

Obsarvatiof) Schedule ar)d Record. Form 5. Verbal (OScAfiSv). This is 
the current revision of an instrument which has a history beginning :n 1958.^^ 
Without question, the various forms of OScAR and the Flanders System of 
Interaction Analysis have been the two instruments used more than any 
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Others by researchers 

The Current revision of OScAR differs sharp;/ from the original, and 
codes inte? actions as sequences, it uses 18 categories— four for pupil 
utterances and 14 for teacher utterances. Since six of the teacher categories 
are dual-purpose, however, there are. in effect. 20 for teacher behavior. 
The Instrunnent provides for recording interactions by pairs of items which 
record either pupil utterance and teacher evaluation or response, or teacher 
question and the teacher*s response to the pupifs response. 

The fac; that both entry to and exit from the interchange may vary means 
that the number of combinations— the number of specific interchanges— is 
large, and although the primary focus of the instrument is substantive. 
sociaUemotional behavior m management is also recorded. 

One aspect of the instrument which seems likely to be important in a 
variety of settings and with a variety of objectives in PBTE is the ingenious 
system for recording interchanges— which not only records a teacher be* 
havior. but also its antecedent. It seems likely that PBTE could use such a 
procedure for coding sequence with a variety of kinds of content. 

Personal Record ot School Experience (PROSE). This ir strument is a 
system for recording the behavior and experience of individual young chil- 
dren.?^ The method used is time sampling. For a •cycle" of five observa* 
tions. on signal from a timer, every 30 seconds the observer records the 
activity of the child at that moment. A series of 1 1 "words" are used to 
describe the activity; words 1*3 describe the adult-child contacts, with 1 
recording the activity of the adult; words 4 and 5 record child-child inter- 
actions. wi*h 4 the target child and 5 the child with whom he is interacting; 
woro 6 re jrds the level of the activity of the target child in the previous 
words, in terms of physical contact, materials, or whether verbal or non- 
verbal communication occurred; word 7 contrasts the sex and ethnic group 
of the target child with the other person; words 8. 9, 10, and 11 describe 
the appropriateness of the activity, the nature of the physical activity (move- 
ment), the nature of other activity (work, fantasy, etc.). and whether the 
activity was structured by a teacher or by a child. 

After five observations (a "cycle") have been coded, the observer records 
the child's context in terms of the general activities of the classroom as a 
whole, other adults, and the instructional content. 

Classroom Observation instrument (COI). This instrument is the one 
which Stanford Research institute has used to represent the major programs 
of planned variation in Headstart and Follow Through. ^* The instrument 
provides for a physical description of the classroom (recorded at the end of 
the day) in which numbers of adults and children are recorded, along with 
physical aspects such as noise level, lighting, furniture arrangement, and 
whether pupils select work groups. The observation begins with a Classroom 
Check List, which records the grouping patterns, materials in use. and adult 
responsibility. The Five-Minute Observation records four items of informa- 
tion for each of 76 frames: Who. To Whom. How. and What. The first two 
identify the initiator and receiver; What includes such items as command or 
request, open-end question, response, or instruction. How includes such 
items as nonverbal, motion, happy, or positive. The information recorded 
in each frame is summarized as a sentence which describes an event, and 
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provision is made for simultaneous or for recurring events. 

Since there are thousands of possible combinations for a sentence, it is 
necessary to combine them into variables for further analysis. 

Other Systems, There are hundreds of systems which have been de- 
veloped, and many are reviewed by Rosenshine and Furst.^* The rr.ost ex* 
tensive collection of instruments, with annotations, is the series of volumes 
by Simon and Boyer.^^ 



When one reviews the major points of this paper, the problems \n class* 
room observation seem overwhelming. We have often approached teaching* 
learning as a simple, linear relationship while it is neither simple nor linear. 
The statistical problems are great; the problem of measuring outcomes for 
pupils is complex; and the measurement of classroom behavior is even more 
difficult. 

Yet there is no alternative to facing these problems, because the most 
important part of schools is what is happening In the classrooms. 
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lnservice« 
Needs Assessment • 
Competency Based 
Teacher Education 



GHbtrt F. Stwarron 



Public schooling probably is the only large-scale enterprise 
In this country that does not provide for systematic updating 
of the skills and abilities of its employees and for payment 
of the costs involved. Teachers, we presume, can acquire 
markedly different one3 through some process of osmosis.^ 

Most persons and organisations who give serious thought to mservice 
education (staff development seems to be the current term) would probably 
agree with the above quote from John Qoodlad. One. of the difficulties in 
inservice education has been that it only provides information to teachers — 
information about what someone else (usually a consultant or college pro- 
fessor or textbook) thought or information shared between teachers. Little 
Of no consideration has been given to assisting teachers in developing and 
practicing new teaching skills In the classroom. And little attention has 
been given to assessing the effectiveness of what the teacher does, and 
what the pupils learn. 

Bob Bhaerman has summarized Inservice education by saying that often 
it has been long on service and short on education. It has been too frag* 
mented, without integrated activities based upon assessed priority needs, 
insufficiently supported by budget, and too Insignificant to leave a marked 
and continuing Impact upon teachers and programs 

Inservice education is a term that has various Interpretations. For pur- 
poses of this paper. It Is defined as the continuous training that a teacher 
needs as a result of being assigned to certain teaching situations. School 
districts have unique needs. Sometimes schools within school districts and 
classrooms within these schools also have unique needs. Few, if any. 
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prdservtce teacher education programs make the claim that their Qraduates 
have aii of the teaching skills necessary to function in any circumstance. 
Therefore, inservice education should focus first on developing those addi- 
tional teaching skills needed to imptement the regular program prescribed 
by the school district. Secondly, the inservice program should focus on 
providing knowledge and teaching skills that result from change in both 
curricula and pupil population. 

For example, in the early 1960 s, teachers who were asked to teach 
**new math" might have needed knowledge and new methodological skills. 
As another example, if a school district adapts a diagnostic, prescriptive 
teaching strategy, then the teacher may need knowledge of diagnostic 
techniques as well as practice in the application of diagnostic procedures. 
Thirdly, inservice education aims at the need for the continuous improve* 
ment of educational personnel 

Inservice education cannot be effective without considering what needs 
teachers have. The position advanced in this paper is that a viable inservice 
education program begins with a needs assessment procedure that requires 
the following: 

1. The identification of teaching skills needed in a particular teaching 
situation. 

2. The development of a procedure for determining teacher needs in 
relation to the identified teaching skills. 

3. A procedure for assisting teachers to acquire and improve his or her 
teaching skills. 

This paper speaks to these three points. It discusses and outlines a strategy 
based on Competency Based Teacher Education (CBTE). 

This paper does not address itself to the knowledge teachers might need. 
It focuses on improving teaching skills. The paper also does not deal with 
the question of who is responsible for teacher growth. While this is a most 
crucial issue, it is not within the scope of this discourse. The paper is limited 
to a needs assessment strategy only for classroom teachers. 

Why CBTE? 

Competency Based Teacher Education is a data-based system for 
training teachers. CBTE precisely specifies that teachers must exhibit 
those competencies assumed to promote pupil learning, and/or demonstrate 
their ability to promote desirable pupil learning.^ Instruction in CBTE focuses 
directly on the development of those competencies thought to promote 
pupil learning. It is the opinion of the author that CBTE is a viable approach 
to needs assessment because it is a data-based system. William Drummond 
points out that in a CBTE program, data are collected and used to make 
adjustments and changes in what and how things are done. Data are not 
collected for punitive reasons/ It is crucial that needs assessment be 
thought of as a procedure for improvement, and not to "get teachers.** 

The basic elements in CBTE provide a way to develop a needs assess* 
ment system that addresses itself to the three requirements for such e 
system mentioned earlier in this paper. Stanley Elam identifies the essential 
elements of CBTE as: 
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1 . Teaching competencies to be demonstrated are role de wed, speci- 
fied in t)ehavioral terms, and made public. 

2. Assessment criteria are competency based, specify mastery levels, 
and are made public. 

3. Assessment requires performance as prime evidence, takes knowl^ 
edge into account. 

4. Student's progress rate depends upon demonstrated competency. 

5. instructional program facilitates development and evaluation of 
specific competencies.^ 

The Elam essential characteristics are perhaps more often thought of in the 
preservice realm. They can, however, be applied to the needs determina* 
tton^inservice area. The position of this paper is that concepfua//y, CBTE 
IS more easily adapted to inservice education than to preservice education. 

Elam's five essentials fit nicely into a needs assessment scheme. Items 
one and two offe' the opportunity to identify teaching skills in particular 
situations. Items three and four provide direction for establishing a pro- 
cedure to determine needs, and the final item addresses itself to the 
improvement of teaching skills. 

The systematic data-based approach utilized in CBTE provides a means 
to tie needs assessment and inservice together with other parts of the 
school program. Robert Houston and Robert Howsam point out that CBTE 
1$ planning in systemic terms, dealing simultaneously with the elements 
that comprise a total system.^ A systematic data-based approach can make 
possible the continuous updating of teaching skills. The use of a systems 
approach* however, requires that specific decisions be made as to what is 
to be accomplished. It also requires that problems in inservice teacher 
education can neither be resolved nor exist in isolation from the total pro- 
gram of the school district. As an example of this, one might consider that 
if the school district is unwilling to provide released time or extra compen- 
sation for a teacher to improve a skill that the district itself says he or she 
must have, then there is little likelihood that the new skill will be developed. 

The (Politics of Assessment 

The assessment of the needs of school oersonnei can be threatening to 
those who are being assessed. The threat may or may not be implied. A 
decision to identify te^^ching skills presumed to be necessary for existing 
and new programs can become a matter of great concern to some individ- 
uals charged with carrying out the program. Organizations representing the 
teachers become concerned that the assessment will be used as a device 
to dismiss and retain teachers, rather than as a device to improve teaching. 

In addition to concern about assessment, the introduction of CBTE at 
the inservice level can become a highly partisan issue. Unfortunately, CBTE 
often elicits an emotional response rather than reasoned consideration. 
Theodore Andrews believes that teachers will demand a significant role in 
any competency based system that includes inservice education.^ A few of 
the political questions to be dealt with aie: Who will determine what 
teaching skills are required? Who will determine which teachers need 
which skills? Who will determine what learning activities are appropriate to 
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develop certain sktlls'> 

The position put forth In this paper is that the design of a needs assess* 
ment system in a CBTE framework must consider, along with the con- 
ceptual strategy, a corresponding political strategy The integration of these 
two strategies calls for a systematic approach to planning and implemen- 
tation 

A Propof td Strategy 

The strategy offered in this paper is an attempt to combine the con- 
ceptual aspects of needs assessment with political considerations. Gener- 
ally, the conceptual notions come from attempts to apply the essential 
elements of CBTE to needs assessment. The political implications come 
from pilot efforts to use these conceptual notions in needs assessment and 
inservice programs. The pilot work has been done m three school districts: 
a 'arge urban district, a middle*ciass suburban district, and a small rural 
dis^ict. 

Cenerally. the strategy begins with the determination of some of the 
teac:^ing skills needed m a particular school district. Once needs have been 
estab shed, assessment procedures are developed to determine where 
teaching personnel are in relation to these needs. Finally, the needs assess- 
ment stra.^y leads to inservice education opportunities. 

There btk two assumptions that provide the foundation for this strategy: 

1 Needs *)ssessment Is based on the improvemf*nt of teaching skills. 

2- Those wi o are to be assessed should be active in the development 
of the asset. 'Kment procedure. 

h^tntHlcatlon of Natds 

The identif ica^ior. of needi i teaching skills is the first step in the strategy. 
By utilizing a CBTE approach. «iaching skills are identified as competencies 
that teachers should have m oi ^er to carry out the program of the school 
district. The school district has to make a decision as to how the teaching 
function should be analyzed (how "ronipetencies should be determined). 
Later, we will describe two approact' es used by two different school dis- 
tricts to determine this. 

The next question to be considerec is: who is to dnaly::e the teaching 
function'^ Most teachers in the field a>'^ inclined to believe that they know 
more than anyone else about what a teacher does. Researchers and those 
who attempt to project what a teacher shouid He usually feel that they know 
more about the role of a teacher. However, inservice teachers are not likely 
to be willing to have someone else analyze the teaching function, and then 
have the administration hold them responsible for these functions. We have 
found that teachers are willing to listen to the advice of so-called experts, 
but they intend to reserve the final decisions for themselves. 

An illustration of a role the "experts" might play would be :n helping the 
teachers to clarify ideas. Suppose that one of the competencies needed is 
the skill of questioning. Here the teachers might be aided by looking at a 
number of alternative questioning strategies which would help to define 
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what types of questioning skills are appropriate, how they might do as* 
sessed, and what types of training activities are availj^ble. 

One approach to developing inservice competencies has been tried by 
the Atlanta Public Schools. A comprehensive revision of their elernentary 
curriculum was carried out in the early 1970*$. Five areas of literacy were 
developed for an pupils: personal, career, aesthetic, social, and intellectual. 
Major ideas, concepts, and objectives were identified for each of these 
areas. In addition to the five a.eas, a basic instructional strategy was 
developed to carry out the curriculum.* The curriculum was developed by 
teachers, community, professional organizations, etc.; then the curriculum 
was piloted in 10 elementa*y schools. After one year of piloting, one of the 
pilot schools was selected to develop competencies thought to be necessary 
to implement this curriculum. 

The teaching staff of the school, university personnel, plus curriculum 
and supervisory personnel from the district office began to systematically 
determine what competencies were needed to implement the curriculum. 
As an illustration. Figure 1 depicts the instructional strategy to be employed 
tn carrying out the curriculum. The teaching staff analyzed the strategy, 
applied their experience in piloting the curriculum for one year, and began 
to determine what skills a teacher needed. The str tegy (see Figure 8.1), 
as you see. begins with **Find what student knows.*' This implied diagnostic 
skills. The five areas of liti^racy referred to earlier indicate that diagnostic 
skills need to go beyond the intellectual into the social, aesthetic, and 
personal dreas. There are, of course, additional competencies needed to 
utilize the prescribed strategy. 

The Atlanta approach might be classified as a curricula approach to 
analyzing the teaching function. They took the basic curriculum v/ith its 
instructional strategy and from this, inferred competencies for teaching. 
The strategy employed in Atlanta was involvement of the teachers (among 
other groups) in developing the curriculum. Teachers then had the oppor* 
tunity to define the teaching skills needed for implementation. While there 
IS still, perhaps, some apprehension on the part of some teachers, this 
should be reduced because of the involvement of the teachers from the 
t>eginning. 

Another effort to develop competencies for purposes of assessment was 
done by the Mentor. Ohio Exempted Village School District. The Mentor 
project utilized differentiated staffing within two elementary schools using 
open-space settings. Within these two schools, professional and parapro- 
fessionai staff members were employed in a role hierarchy (i.e., master 
teacher-instructional strategist, staff teacher, assistant to the teacher, and 
intern). The project focused attention on the need to enhance instructional 
opportunities for pupils by developing a systematic approach for continuous 
staff growth based on individual as well as differentiated staff needs. 
Specifically, the project focused on specifying competencies for teachers 
in a differentiated stc':eHi in pattern. These competencies were to be used 
in an assessment proceduie to determine what types of staff development 
activities were necessary to enable the teaching staff to develop and main* 
tain the desired level of proficiency. 

The strategy utilized in Mentor was initially to have the personnel involved 
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react lo an infitrumant developed to measure perceptions of competency 
levels within each school The statements utilized in the instrument were 
drawn from the literature on differentiated staffing and from current opera* 
tional job descriptions. Siaff memtMirs were asked to react to the degree to 
which the statements were operational in their buildings. They were also 
asked to indicate the importance of each statement. One of the statements 
on the instrument was "Teachers in my building show evidence of successful 
planning of learning activities.** Statements such as this provided a basis of 
discussion which led to the specification of competencies for a differentiated 
staffing pattern.'^ 

The Mentor appfoach can perhaps best be described as a speculative 
approach to analyzing the teaching function. The instructional staff, with 
some structure present in the instrument, speculated on what teachers do 
within a differentiated staffing pattern. The political strategy is present 
because those who are to be affected were making the decisions. 

There are. of course, other approaches to developing competencies for 
inservlce teachers. The two illustrations were used because of the attempt 
to put together conceptual and political strategies. In both cases, once tho 
competencies were Identified they provided the baseline data that describes 
where the school district wants to go and what a teacher should be able to 
do. 

It is interesting to note that in both the Atlanta and Mentor efforts, neither 
group identified as a teaching competence any of the strongest variables in 
the relationship between teacher behavior and student learning identified 
by Barak Rosenshine and Norma Furst.'' This would indicate the lack of 
inputs by researchers into determining competencies for teachers in the 
two districts. Consequently, we would suggest that part of any identification 
of teaching skills be preceded by opportunities for consideration of research 
on teaching. Most teachers have not had the opportunity to consider the 
theoretical and research aspects of teaching. Although the decision for 
determining the specific competencies should be the responsibility of 
teachers, they need to be exposed to the study of teaching. 

Assesetnp tiM NMde 

The assessment of needs requires that some type of scheme be de- 
veloped that can describe where a teacher is in terms of the school dis- 
trict's requirements and can also provide an opportunity to chart a direction 
for improvement. In developing assessment procedures, our experience 
has been that you begm with the assumption that persons who have a 
certificate and have been employed by a school district are competent in 
the opinion of those who granted the certification and those who employ. 
This assumption may be odious to some readers; but nevertheless, it is 
difficult to start with the assumption that some teachers are competent 
and others incompetent because cnmpetenca without some data base is 
a relative thing. A teacher with a certificate and a Job has as much claim to 
competence as anyone else, it also is, in our opinion, unwise to start with 
the proposition that assessing needs is a basis for finding out who is and who 
is not competent. It has been our experience in the three school districts 
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that when we start from o base that everyone is presumed competent, there 
IS less reluctance on the part of teachers to participate. 

W. Robert Houston uses the illustration of the violin when discussing 
competence. He points out that the violin soloist at the symphony must be 
able to read music, properly handle the bow. tune the instrument, and have 
a stage presence But so must the beginner m the seventh grade concert. 
The differences between the violin soloist and the seventh grader he m 
the criteria which are acceptable for an adequate performance. 

Houston then points out the parallel m teaching. The beginning teacher 
may perform adequately m asking higher order questions, establishing 
sets, and writing criterion-referenced objectives, but might not be able to 
integrate those skills and employ them in given circumstances The be* 
ginning teacher may be adequate the first year on the job, but after a tow 
years of experience, she should be able to integrate the skiMs and employ 
them appropriately. In the Mentor model referred to earlier in this pap^r. 
the Master Teacher instructional strategist certainly should be able to more 
effectively utili;re teaching strategies than the staff teacher. 

Thd differences m levels of competence and the reluctance by teachers 
to be associated with incompetence led us to utilize a continuum of pro- 
ficiency for each competency, the definition of proficiency beiny to advance 
or move forward By utilizing the continuum, we have the opportunity to 
allow movement through inservice education toward new levels of pro* 
ficiency. Theoretically, one can always become more proficient. The idea 
of continuous improvement of oni^'s skills is difficult for any organized pro- 
fession to be against. Point one on the continuum can be the position where 
one enters the profession or where minimum proficiencies are exhibited. 
The last point of the continuum can be the ultimate in teaching or the place 
where a teacher uses special skills in particular situations. 

At each point on the continuum, a set of criteria must be developed that 
IS accepiable at that level A special set of criteria ts needed for the violin 
soloist and a special set for the seventh grader. Figure 6.2 illustrates how a 
continuum of proficiency might look. 

The criteria or indicators of proficiency help us to operationahze the 
competency. They provide the precisn criteria and conditions under which 
the competency will be judged, in a needs assessment system, the criteria 
not only help in the identification of where the teacher is. but they will aid 
m setting directions for improvement. For every competency, representa* 
tive indicators of proficiency should be developed. It is the position of this 
paper that they should be developed along a proficiency continuum. We use 
representative indicators because, in our opinion, the same indicators are 
not necessarily acceptable m all situations. 

Teachers also feel that because of variation in conditions, the same 
indicators are not appropriate for every situation. For example, learning 
Outcomes being related to pupil entry behaviors may not always be possible 
or desirable. Available data on pupil learning behavior may be limited only 
to the cognitive. Or the lesson being taught may focus on experiences that 
someone thmks are desirable for pupils to have, if the purpose is to 
experience something, then pupil entry behaviors may not be important. 

Chances are that the indicators of proficiency may be appropriate for 
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iomo teachers in some classes, but they are certainly not appropriate for 
evury teacher in every situation Therefore, it Is our feeling that those In- 
volved in the needs assessment should have the opportunity to negotiate 
additional indicators of proficiency either for purposes of assessment or for 
determining appropriate directions for his personal inservice education. 

Negotiation becomes a critical feature of the proposed strategy. Whom 
the teacher negotiates with will be discussed in the succeeding paragraphs. 
But the politics of negotations gives the teacher the opportunity to have his 
views plus his unique teaching situation considered. Conceptually, there 
are. at this point in time, few people who are willing to state categorically 
that they can specify all of the conditions and criteria under which one can 
be stamped compefenf* 

The negotiation of proficiency also allows for dialogue amotig the teacher 
and those who participate in the assessment process. The beginning teacher 
can be assisted m considering what are the most appropriate skills to be 
developed immediately white the experienced teacher can be helped to 
establish some sort of priority. Moreover, it is also useful to have someone 
else's opinion in determining where one ranks on a proficiency continuum. 
Finally, negotiation makes the individual being assessed active in the entire 
needs assessment effort 



WhoAttMMt? 

Negotiation implies that self^assessment be part of any needs assess- 
ment procedure. Our experience has b^c * that most teachers express a 
desire to use self-assessment for one ot if^e following reasons: (1) They 
hypothesize that they are not likely to change their performance unless 
they see a discrepancy between what they want to achieve and what they 
are actually achieving. (2) They feel that they know better than anyone 
else about their particular teaching situation 

John McNeil and James Popham point out that there is a tendency when 
using self-assessment for teachers to overrate themselves. Further, there 
are negligible relationships of self-assessment with other criteria such as 
student ratings and measures of student gain. McNeil and Popham also state 
that most teachers seem to criticize the superficial aspects of their teacn- 
ing, personal mannerisms, appearance, etc., rather than the relevant 
aspects of their work. 

Still the political expedient remains. We believe that at this point in 
time, self* assessment will probably have to be part of any assessment 
procedure that carries teacher approval. However, to be effective in self- 
assessment* teachers should be trained to focus on the relevant aspects 
of tneir work, especially in judgirty the consequences of their behavior on 
pupils Furthermore, teachers must have opportunities to practice these 
skills. There i$ competence in the application of self-assessment skills just 
as there is m asking questions. If self assessment skills plus proper atti- 
tudes toward needs assessment can be built, then the acceptance of a 
needs assessment system can be enhanced. It is our opinion that self- 
assessment and the political strategy are complimentary. 
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A needs assessment system also should be related to the administrative 
hierarchy of the school district. There should be suiiitt type of subordinate- 
superordinate relationship. If the teacher is in the subordinate role, then the 
role of superordinate might be played by a principal, supervisor, or In a 
differentiated staffing pattern by a team leader. The superordinate, while 
perhaps posing a threat in some instances, does have some responsibility 
for seeing that programs are carried out. If needs assessment Is to be based 
on improvement, then the superordinate has responsibility for this, being in 
a position to make the inservice effort more responsive to the teachers* 
needs, it is the position uf ihis paper that through the use of indicators of 
proficiency, with the teacher and superordinate negctlatinr the InJiwaiOiS, 
there is less possibility of threat to the teacher. The possibi»ity of dialogue 
for improvement and understanding is also expanded. 

As an illustration, suppose that a competency for a teacher is in the area 
of diagnosis. Teacher A has no particular skills In diagnosing pupils. 
Teacher A and the principal agree that the teacher should focus first on 
diagnosing the intellectual abilities of pupils. They agree that ono of the 
criteria to be used in assessing intellectual abilities is the use of teacher- 
made diagnostic tests for groups and individuals. In oroer to become more 
proficient. Teacher A is to develop a skill in building tests that can Identify 
where pupils are on specific skill continuums in reading and mathematics 
Inserv'ce activities should then be recommended by the principal that will 
help Teacher A to learn how to construct and administer diagnostic tests 
in reading and mathematics. Once the teacher develops the skill and 
utilizes it in the classroom, then there would be opportunity to develop 
further diagnostic skills that would vwniinue to make the teacher more pro- 
ficient. 

We believe that teachers and administrators can work together in Iden- 
tifying needs if it is for purposes of improvement. Obviously, there are the 
human requirements of trust and consideration that must be present in the 
relationship for this or any other working relationship to succeed. We are 
hypothesizing that if a subordinate and a superordinate agree on an area 
of improvement, then precisely define what the improvement will be, there 
is a good possibility that improvement can be assessed by both parties. 
Agair it is also important to consider that the administrative hierarchy 
be responsive to the needs of the teacher. A hierarchy is likely to react more 
readily if it is represented in the process. 



Summary 

This paper has presented a proposed strategy for developing and utilizing 
a needs assessment strategy based on Competency Based Teacher Educa- 
tion. Tho strategy takes into account both conceptual and political con- 
sideration;. It comes from attempts to combine some of the conceptual 
elements of CBTE with pilot efforts to use these conceptual notions in 
school districts. Some of the suggested procedures are as follcvvs: 

1 . Teachers should make the final decisions on needed competencies 
to perform effectively in a school district. However, teachers should 
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have opportunities to consider the theoretical aspects ot teaching, 
including research findings pnor to making decisions about needed 
competencies 

2 The assessment procedure should begin with the assumption that 
needs assessment is for the improvement of teaching Therefore, we 
begin with the notion that teachers are competent, and that the 
purpose of the needs assessment is to help them become more 
proficient 

3. A continuum of proficiency for each competency should be developed 
in order to a'low for continuous teacher growth The mdicators of 
proficiency on this co'^t'nuur''* snould be negotiated by the assessor 
and the teacher. 

4 Sell assessment should be part of the needs assessment procedure. 
However, self-assessment skills need to be developed m order to 
make this a successful venture. 

5 The administrative hierarchy of the school district should be involved 
m the assessment procedure. 
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9 Assessing Teacher 
Competence 



Grata Moc ln« 
David a Barlinar 

Assessment of teacher performance may be conducted for a variety of 
reasons, such as selection for entrance into preservice training, idtntifi* 
cation of the types of training most needed by a preservice and/or mservice 
teacher, awarding of a certificate to teach, and research into effects of 
teacher use of particular skills. 

For the purposes of this paper, we are focusing upon assessment as a 
means for selecting training options. We have made this choice foi iwo 
reasons Firsi. tne body of knowledge in teacher effectiveness research at 
the present time is loo limited and too subject to question to justify using 
assessment data for selecting preservice candidates or granting licenses 
to teach/ We do not as yet know enough about which aspects of teaching 
are essential for affecting student performance positively to use specific 
levels of teacher performance as "go ' or "no go ' criteria for entry into or 
promotion within the profession 

Yet much is known about measurement of teacher performance. Large 
numbers of evaluative studies report describing typical levels of teacher 
performance before and after training.^ To ignore these data while waiting 
for completion of effectiveness research is. m our opinion, as great a mis- 
take as using them for hiring or firing purposes. Thus, our second reason 
for concentrating upon assessment for training purposes is to il'uStrate how 
the current ctate-of-the-art in the study of teaching can be productively 
applied to the design of a self-improvement program for an individual teacher 
or a group of teachers 

Within the spectrum of performance based teacher education. <Averal 
training approaches have been developed. The assessment techniques 
described herein illustrate two diverse points on the continuum of ap- 
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proaches The skill diagnosis approach represents a behavionstic view of 
assessment, the inquiry approach, a ^umanlstlc view Three critical dif- 
ferences in the approaches are (a) the data base from which decisions are 
made regarding the need for training. (' . the individuals responsible for 
designing the assessment event, and (c) the type of evaluative feedback 
provided 

Both approaches are under development at the Far West Laboratory for 
Educational Research and Development. Work on the performance tests of 
teaching is m cooperation with Nattonal Teacher Corps and Corps sites at 
Kansas State Teachers Colleg<e. Empona, Kansas. Buffalo State University 
College. Buffalo. New York, and the Southern Consortium of uolleges and 
Universities. Mobile. Alabama. The inquiry approach is being developed in 
cooperation with the Oakland Unified School District. Oakland, California, 
under the auspices of the Bay Area Teacti<»r Training Complex, sponsored 
by a grant from tr^e U S Office of Education. 

A Skill Diagnosis Approach 

The Skill diagnosis approach to analysis of competency m teaching com- 
petency IS based on a belief that teacher trainers and researchers need to 
introduce a medical model into the way they examine teachers and their 
teaching practices Withm the medical profession, the model requires the 
monitoring of a numbei v\ potentially important systemic indicators, such as 
heart rate, patellar reflex, and blood pressure. Then clinical knowledge 
about the client is developed — his or her job requirements, temperament, 
previous hiMory. etc On the basis of these data, if certain indicators are 
out of tolerance, individualized prescriptions are made (e.g . lose weight, 
quit smoking, take medication four times a day). 

Teacher trainers and researchers also need to develop ways to monilut 
the systemic functioning of their clients, perhaps checking their warmth 
instead of their temperature and their response to pupils rather than to 
light. Then, with clinical insight derived from information about the teacher's 
experience and interests— and perhaps the age and social class of the 
students to be taught— individualized prescriptions could be prepared, such 
as "Take Module 014 from the Houston Module Bank for learning rein- 
forcement techniques to use with young inner-city children." 

How to find the indicators for diagnosis and prescription of this kind is 
the concern of the sk:l! diagnosis approach to assessment. 

Knowtng "What" and Knowing "How" 

Multiple choice or essay tests are easy to develop and grade, can some- 
times even be reliable, and cost very little per client tested. One can readily 
assess knowledge about what discovery-teaching is, what one should do 
with obscene notes, what the definition of contingent reinforcement is, etc. 
This knowledge of 'what may be important, but its relationship to knowing 
how to teach is quite indeterminate. A recent view correlating a highly 
reliable test of knowledge about teaching with various measures of teach- 
ing performance informs us that virtually no relationship exists between the 
two variables^ Indicators for examining teaching should not come from 
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papor-and-pencil tests which measure what is known. The indicators must 
be based on how teachers perform. 

Obterv«tton In Natural Envfaronmtnta 

Some believe, along with the many developers of observation instruments, 
that observations of how teachers perform in their classrooms is the way 
to assess teaching competency. But Inclass observations might be very 
much like the annual visit to the classroom by the principal — an exercise in 
futility. In most philosophic systems, the 'good" teacher is one who varies 
his or her behavior depending upon the child, the learning objective, the time 
of year, and a host of other variables. All these are influences on teacher 
behavior designed to make the behavior uristabie in situ. Thus observation 
of teachers m natural environments may be an inappropriate way to develop 
reliable and valid indicators of teaching performance. 

Parformanca Tatta 

In performance tests of teaching, where a controlled environment exists, 
indicators for skill diagnosis may be found that are similar in type to those 
used by our medical colleagues, in these diagnostic tests, a teacher trainee 
receives a statement of two or three objectives, some written material on a 
topic, and sample test items. The teacher is told to prepare to teach five or 
six students for 20 minutes. The teaching performance is video- or audio- 
taped. Frequency counts or ratings on as many as 50 variables are com* 
pleted. 

In order to develop such tests, three important elements must be 
considered: 

Choosing variables. Thus far, the literature on teacher effectiveness 
(such as it is) along with the value Systems of the participating researchers 
and teacher trainers have directed the selection of variables. Variables 
also have been picked to match those included in existing teacher training 
materials since it is important to examine the pharmacy, i.e., the Mini- 
courses, texts, protocols, simulations, etc., available for use in responding 
to a skill diagnosis before writing a prescription. In the performance tests, 
we try not to be like the doctor who diagnoses terminal cancer and offers 
no cure. We assess first if there is a cure, i.e., a training module, and 
then proceed to diagnose by examining some indicator to see what, if any, 
training need exists in the Curable area. 

Evaluating performance. The teacher's performance Is evaluated 
against, (a) norms which have been accumulated from other trainees, 

(b) the performance of master teachers attempting the same lesson, and 

(c) criteria we have set for performance in a particular area. In light of each 
or all of the^c sources of information about teaching performance, a diagno- 
sis is made. Feedback is provided to the trainees on each diagnosis. For 
example: 

Variable 1 : Teacher gives specific praise or encouragement 

Variable Desnription: The teacher praises or encourages students and 

clearly states why praise or encouragement is given. Example — "That is 
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a good answer because . " 

Performance Evaluation 

1. ) This evaluation occurred JL times in your teaching. 

2. ) Based on norms and our criteria. to 12^ times is desirable. 

3. ) The diagnosis to be made is. 

Performance is within limits 

Performance is too high 

^ Perffjun is too low 

Prescriptions: if performance is too high, see Nos. 2. 4, or 31 . 

If performance is too low. see Nos. 30. 1 . 32. 

This form of feedback is continued for as many variables as desired. In 
each case, the trainee receives descriptive information and definitions of 
the variable, his actual performance, estimates of desirable performance, 
and prescriptions for remediation. 

PrescripUof). The numbers in the prascripUon list refer to training 
materials which exist in the "pharmacy " For example. No. 30.1 may be the 
code for a lesson in a Minicourse that focuses on contingent reinforcement; 
No. 32 may refer to a protocol videotape that demonstrates Ino concept of 
"because extensions" in communication, or the trainee may be referred to 
a chapter in a text on the effects of praise and blame on oerformance. If 
the trainee's performance was too high, that is. if performance exceeded 
the levels considered desirable, the trainee might be directed to materials 
which teach about habituation, since praise will become ineffective if too 
much is given, because people tend to disregard it or question its genuine- 
ness. 

For any teacher variable of interest, if an out-of*tolerance situation is 
detected in a performance test, one or more prescriptions can be recom^^ 
mended, in this way these tests are as diagnostic as the physician s exami- 
nation. An individually prescribed program of training may be developed 
which allows a teacher to concentrate on areas where training will be most 
useful and skip areas where performance competency has already been 
demonstrated. Preliminary study of these performance tests has shown 
them to be reliable. Stability of performance seems to hold ove: parallel 
lessons and for the same lesson taug*^t at adjacent times. This means that 
the performance tosts. when thoroughly developed, can be used not only to 
plan training but also as objective measures of the effectiveness of compe- 
tency based teacher training programs. 

An Inquiry Approach 

All approaches to assessing teacher competence share at least one 
common purpose— the improvement of instruction. The two approaches 
discussed here share a common orientation to attainment of that goal — t. 
view that formative evalualiun of teacher effectivent»s? is the most viable 
route. The inquiry appiodch differs from the diagnostic/prescriptive ap- 
proach in actively pursuing the following additional purposes: 
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1 To provide a supportive climate in order to maximize change in 
teacher behavior. 

2. To indiv)dur\ii2e the explicitly stated criteria on which assessments 
of teacher competence are based. 

3 To provide teachers with a strategy for continuous self-assessment 
and self-improvement. 

The inquiry approach to be described here had its roots in an analysis of 
the process of inquiring into One's own teaching behavior proposed by 
Flanders* 1his analysis was directly related to assessment of teacher 
competence m a later paper on the California Stull Act.*^ which proposed 
teacher inquiry projects as one method of asses<;ment. The comparative 
procedure which is an essential element of this approach was tested and 
found to be effective in getting teachers to be more specific m identifying 
training needs and more interested in continued skill training.** 

Procedure for Inquiry Projects 

A teacher embarking upon an inquiry project for the purpose of self- 
assessment would engage in the following steps: 

1. Identify one or more instructional objectives and plan two alternative 
lessons to achieve the objective(s). 

Complex*^*; of plans may vary here. An instructional objective may 
be Simple and immediately attainable, such as "to identify correctly 
Six words as nouns, given a list of ten nouns and verbs. " The objective 
may be more complex and long-term, such as. "to increase pupil 
Skill m listening lo and reacting to each Other's commeots." The 
alternative lessons for achieving the instructional objective may 
involve simple variation in use Of One particular teaching skill, such 
as praising pupils for every correct answer in one lesson and praising 
at intervals for attentive behavior in thA other lesson. Variation in 
instructional strategies would be a more complex approach to plan- 
ning alternative lessons, e.g.* using an inductive discovery strategy 
in one lesson and a deductive strategy in the other. (The illustrative 
inquiiy project in the appendix has short-term objectives and in- 
CiuHes variation in the use of a particular skill as well as variation in 
instructional Strategies.) 

2. Decide what data to collect for evaluation of the effectiveness of the 
two lessons. 

OMa on the interactive bchaviur of the teacher will usually be used 
to determine whether the planned variations in the two lessons did 
in fact result in observable differences. Data on the interactive 
behavior of pupils may be used to determine if instructional objec- 
tives were achieved. Samples of pupil work or pupil attitude invento- 
ries may also be used for this purpose. Interactive data may be 
collected by use of a standard category system* but usually the 
teacher wili need to adapt such a system to highlight the differences 
he/she plans to create, it is also possible for the teacher to design 
his/her own simple category system. (The sample inquiry project in 
the appendix utilized the Flanders Interaction Analysis System with 
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a time-line display and with the addition of special subscripts. Addi- 
tional required data for thi3 project were pupils' papers to be 
evaluated on the basis of divergent thought production ) 
3. Teach the two alternative lessuns to two different groups of pupils 
and collect the necessary data 

Lessons may be audiotaped or videotaped for later playback and 
analysis of inter active behavior Another possibility, if a simple tally- 
ing of a specific behavior is the only interactive data to be collected. 
IS to have another teacher observe the lessons and code behavior 
Involvement of colleagues in one's inquiry projects generally oper- 
ates to increase the supportiveness of the climate, it is recom- 
mended that teachers try to get feedback from their peers at one or 
more stages of the inquiry process (For the sample inquiry project 
in the appendix, the lessons were videotaped for later coding.) 

4 Compare and analyze the two alternative lessons, and evaluate their 
effectiveness in attaining the desired objective(s). 

One cr;teriun for assessment of teacher competence is that the 
teacher can demonstrate planned variation in behavior. Another 
criterion is that the teacher can achieve the instructional objective 
in at least one of the two alternative lessons. A third criterion may 
well be that the teacher can plan, conduct, and learn from a project 
in which he studies the effects of his own behavior on pupil outcomes. 
(In the illustrative inquiry project in the appendix the teacher was 
able to demonstrate behavioral differences. Samples of pupils* work 
are not available for us to judge whether the instructional objective 
was achieved.) 

5. Continue to test additional alternatives or work to improve skill in 
using the more effective procedure. 

One result of the inquiry project approach may be that the teacher 
generates additional variations to improve the attainment of the 
original objective, or other similar objectives. This could lead to 
improvement of instruction as the teacher identifies more and more 
effective alternatives. Another possible result of the inquiry project 
is that the teacher finds a particular skill or strategy to be slightly 
more effective, and decides to improve his ability to use it. This could 
lead to improvement of instruction as the teacher engages in further 
skill training. (The teacher who conducted the sample inquiry project 
in the appendix decided to learn more about teaching strategies that 
encouraged divergent thinking and began studying the synectics 
model presented by Joyce and Weil.)^ 

Anticipated Outoomtt 

The inquiry approach outlined above has not been thoroughly tested as 
a method for assessing teacher competence. Materials for tramirtg teachers 
to use the method are currently under development* and will soon be field 
tested. 
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In preliminary fieW tests there has been some indication that the pro- 
cedure of planning, teaching, and comparing two alternative lessons for 
purposes of assessment operates to provide a more supportive climate 
for change. Almost all teachers following this procedure are able to exhibit 
some observable differences in behavior and to identify differences in the 
effectiveness of the lessons. In most instances they judge one lesson to 
be more effective than the other, even though neither was totally effective 
in achieving the stated objective. Thus the comparative method is expected 
to result in some feelings of positive accomplishment and to provide some of 
the self-confidence necessary for exploration, assessment, and change of 
behavior. 

The inquiry approach can serve to individualize the criteria on which the 
assessment of teacher competence is based. A teacher engaging in an 
inquiry project can identify the specific criteria for assessment in relation 
to particular problems of the class he/she is teaching. For this reason, the 
teacher will have more reason to view assessment as a process that con- 
tributes to improvement of instruction in his/her classroom, yielding useful 
knowledge about what is effective for the individual teacher in a specific 
setting with particular pupils. 

Teachers who have learned the inquiry approach will have developed a 
strategy tor continued self-assessment. They need not wait for an outside 
assessment to determine how to improve their instruction. Furthermore, 
teachers using the inquiry approach will be practicing ways of varying in* 
struction. There is some indication that increased variation in itself could 
lead to a general improvement of instruction.* 

While there iz certainly no available evidence to suggest that inquiry 
projects will single handedly solve the problems attendant on the current 
demands for assessment of teacher competence, there do seem to be 
sound reasons for thoroughly testing the effectiveness of this approach. 

Summary 

Further development and refinement of the two assessment approaches 
presented in this paper will contribute much to the quality of teacher training. 
As the research base regarding teacher effectiveness expands and im- 
proves, assessment also can expand and improve. Teaching strategies, 
skills, characteristics, etc. that are shown to have positive effects upon 
student learning can be added to the testing and inquiry packages. Ulti- 
mately, as much more inturmation is available regarding teacher effective- 
I'less. as the psychometrics of measuring teacher behavior are understood 
and resolved, and as the interface between performance based training and 
the in-school teaching environment becomes more compatible and com- 
plementary, we may be ready to use assessmerti fur such purposes as 
entry into training and certification to teach. In the meantime, both research 
into teacher effectiveness and time spent in training can be improved 
through the use of diagnostic and inquiry assessment strategies. 

FOOTNOTES 

* M D Gall. Th9 ProMm of "Student Achi0v9m9nr in /?#S9«rc/9 on factwr Education 
Cftorfs (Paper prasontod at aEAA. Naw Orleans. Faixuary. 1973). 
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•1V»ortne fr ans^>rmrn(J me ProduCf (Paper presented at the AERA Cnicago 1974) 

• Rosenshine and f urst Research on Teacher Performance Criteria 



APPENDIX 
Sample Inquiry Prolact 

Many teachers find it difficuU to get pupils to thmk and write creatively. 
This teacher decided to test two methods of stimulating pupHs to think and 
write about the future The specific objective of these lessons was that 
p«ipils would write a divergent prediction about "The Family ot the Future." 
In both lessons the teacher wanted to encourage pupils to listen to each 
other s Ideas m the behef that this would stimulate additional thinking. 

Inquiry 0b|ective8 

To compare two different ways of usmg pupil ideas as methods of en- 
couraging pupils to listen to each other. 

To compare two instructional strategies (inductive thinking versus ana- 
logical thinking) as methods of stimulating pupil thinking and writing. 

At9Mtfnenl QuMtiont 

A Was the teacher able to vary the two lessons as planned? (See 
interactive data collected ) 

1 . What differences were there between the two teaching methods 
planned"^ 

2 Did these differences show up on the time line? How'' 

3. What differences in pupil part'Cipation were planned? 

4. Did these differences show up on the time line? How? 

B. Which lesson was more effective m achieving the instructional 
objectives'' 

1. What additional information (besides interactive data) did the 
teacher collect m order to make this judgment? 
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Ifl Teacher Competency 
lU Research 



Barak Rosenshlna 

The purposes of this paper are to review current research on teaching 
skills and to suggest procedures for future research which can take place 
as part of competency based teacher education. 

There ar^ two action suggestions within this paper The first is the sug- 
gestion that each institution engaged in competency based teacher educa- 
tiof* conducts one experimental study per year involving teacher skills and 
student achievement. The second suggestion is that each institution pro* 
duces one interpretative review on a teaching skill per year. Details about 
how this might be done and possible topics for study are contained within 
this paper 

State of Research on Teactiing 
Competencies and Pupil Achievement 

Most papers on teacher education contain the embarrassing recognition 
that the present scientific base for teaching and teacher education is 
primitive That is. the number of studies which have looked at both teacher 
behavior and student outcomes is embarrassingly small. A diligent search 
wjIi uncover less than a hundred studies. The quality of many of these 
studies is questionable. The results are most useful for suggesting future 
research; the results of these studies are not sufficie tly strong or clear 
to direct teacher training practices or certification or evaluation of 
teachers. 

In contrast, there is no lack of research on how to itain teachers to intro* 
duce a lesson, ask questions at higher cognitive levels, redirect pupil 
responses, praise students, or encourage student talk. Nor do we lack for 
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lists of teaching skills. What is lacking, however, is research demonstrating 
an expennfientai or correlational connection between these skills and 
measures of change in pupil achievement and affect. 

Statements on the lack of resedich may be surprising to many readers. 
Readers are invited to check these claims for themselves. Consult a text- 
book in educational psychology, teaching, or subject area methods and see 
how much research exists for oft-proclaimed teaching procedures such as 
matching teacher style and pupil style, asking questions* using behavioral 
objectives, promoting student talk, and motivating students, in these areas, 
as in others, I find Only a few studies which took place in classrooms, and 
these have yielded equivocal results. 

I believe that there are insufficient studies to date, to permit an evaluation 
of the promise of this research. Currently, an important task is to generate 
more and improved research so that the promise of this research and direc* 
tions for future study can be evaluated more adequately. 

Research Findings to Date 

There have been very few attempts to summarize the current state of our 
knowledge on teaching competencies, in one review, the correlational 
research on some 25 teaching competencies is presented.^ Nine of these 
25 variables were selected as the most promising for future research on the 
basis of results obtained to date. A subsequent review supports, in general, 
the Rosenshine-Furst findings and conclusions.' It Should be noted that 
these nine "most promising** variables were selected from a review of the 
correlational literature, and future experimental studies may not support all 
of these correlational results. 

The variables below were listed in the 1973 summary, and more details 
are available in the 1971 volume. Those nine variables were: 

1 . Clarity of teacher *s presentation 

2. Variety of teacher-initiated activities 

3. Enthusiasm of teacher 

4. Teacher emphasis on learning and achievement 

5. Avoidance of extreme criticism 

6. Positive responses to students 

7. Student opportunity to learn criterion materia! 

8. Use of structuring comments by teacher 

9. Use of multiple levels of questions or cognitive discourse 

These nine variables are promising suggestions for future research. They 
cannot be used as checkpoints to assess teacher competency because the 
research to date is too Incomplete. 

At first glance, the above list of most promising findings may appear to 
represent educational platitudes. Their value can be appreciated, however, 
only when they are compared to the variables which have not shown signlfi- 
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cant or consistent relationships with achievement fo date. These variables 



are: 




1. 


Non-verbal approval 


2 


Praise 


3. 


Teacher warmth 


4 


Ratio of a// indirect to a// direct behaviors ( I /D ratio) 


5 


Number of different teacher behaviors (flexibility) 


6. 


Number of questions 


7 


Questions classified into two levels 


8. 


Teacher talk 


9 


Student talk 


10. 


Teacher experience 


11. 


Teacher attitude toward subject matter 



Against these above two lists* there is a much larger list of teacher com- 
petencies for which training is currently given. For most of these variables I 
have been unable to locate more than one or two classroom-based studies. 

These variables include: 



1. 


Probing of student responses 


2, 


Selection of tasks to fit readiness level of pupils 


3 


Motivating pupils' attending behaviors 


4. 


Establishing and maintaining rapport 


5. 


Providing feedback 


6 


Conducting individual activities 


7. 


Presenting information 


8. 


Explaining subject matter 


9. 


Giviftg examples 


10. 


Using inductive strategies 


11. 


Using deductive strategies 


12. 


Pacing 


13. 


Responding to student answers 


14. 


Preventing discipline problems 


15 


Giving homework 


16 


Solving problems 


17. 


Developing pupil self-concept 


18. 


Developing learning to learn skills 


19 


Developing acceptance of responsibility 


20 


Selecting instructional strategies 


21. 


Involving students in self-evaluation 



The state of the art, then, is one of a lack of research. We simply do not 
have enough studies about the relationship between teaching competencies 
and pupil achievement. 
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Teacher Questions 

The Situation is more complex than merely a lack of research. There ts 
also a need to tntebiate research findings into some meaning for teacher 
educators. At times this is a difficult task because of the scarcity of studies 
and the difficulty of comparing studies. These difficulties are illustrated in 
the mini review on teacher questions presented below. 

Training preservice teachers to ask a variety of types of questions is part 
of any teacher training program. In some programs, instruction on types of 
questions is given through assigned readings or through worksheets and 
examples. In other programs, trainees receive additional practice in a 
microteachmg setting and receive feedback on the number of questions they 
asked which were factual, analytic, divergent, convergent, comprehension, 
synthesis, evaluative, application, comparative, inferential, or any of the 
bewildering variety of categories into which questions have been classified. 
The objective of the training, of course, is to increase the teacher's use of 
a variety of nonfactual questions. Tens of training studies have been con- 
ducted, and almost all show that a teacher's use of a variety of higher 
cognitive level* questions increases as a result of training. 

Although the results of training studies are straightforward, the results 
are not so clear when one inspects correlational and experimental studies 
involving teacher questions and student achievement. {To the best of my 
knowledge, interpretative reviews of teaching behaviors and other student 
outcomes such as student attitudes or beliefs about oneself have not been 
written. Such additional reviews are urgently needed, and readers are in- 
vited to consider conducting such reviews.) 



Correlational Studies on Questioning 

Some 15 correlational studies were found which contained classifications 
of teacher questions and measures of student gain. Fifteen studies is an 
unsatisfactorily small number when one considers that there are at least 
five grade level groupings (i.e.. primary, intermediate. Junior high, senior 
high, and college), at least tour subject areas, and at least two types of 
cognitive outcome measures (factual and comprehension). This alone 
yields 40 separate cells—and there are only fifteen studies, and the quality 
of many of them is less than desired. 

Fifteen correlational studies on teacher questions and student gain 
reported in the last 15 years of research I Without any trouble, one could 
easily find 50 books, chapters in books, and journal articles, which claim 
to teach how to ask questions. In the two volumes on Resourcos for Per- 
formance-based Education, there are 26 modules on how to ask questions— 
and an equal number of filmstrips. movies, audiotapes, and multimedia 
kits.^ We are instruction rich and data poor. 

DItferencea in definitions. Even with only IS studies, there are problems 
in interpretation because variables with the same title (e.g., factual 
questions) are defined nnd coded differently in different studies. Although 
investigators usually classify questions into two types, many questions 
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Which are cuissifiiul .r> hioh# ♦ \%i^e\ questions (or open or interpretative 
questions) D> one «nvestigator have heen coded as lower order questions' 
by another investigator For example, questions which require a pupil to 
explain how he got an answer, or questions which require the answer to a 
thought problem, have been classified as factual by one investigator and as 
higher levei qut^st'Ons bv another 

Even if one knew the precise definitions each investigator used to classify 
quest'ons. we still would not know how many explaining Questions, for 
example, were classified among the higher level questions by one mvesti* 
gator and among the lower level by another Because of these problems, 
any comments on the overall results must be treated with caution 

Results of correlational studies Of the six studies m primary grades m 
which questions were broken into two types, significant results favoring use 
of higher level Questions were not obtained m any study These results held 
for reading, mathematics, science, standardized tests, and author- 
developed tests In one study, open Questions with a nonacademic focus 
were significantly negatively related to math gam m the first grade (but not 
i.s the third grade for similar lower class pupils) In another study, questions 
which asked children to give a personal preference or respond from per- 
sonai experience were negativeiv related to gam m reading, particularly for 
middle class children (in the same study, these results did not hold for 
mathematics gam i Thus, the overall results for open questions m the 
primary grades are either nonsignificant or negative in four of the above 
studies there was no significant relationship between the frequency of 
factual questions and student achievement in the fifth study there was a 
positive, significant relationship tn the sixth study the result was positive 
and Significant for academic factual questions; the correlations were 
negative and significant for nonacademic factual questions Thus, the 
overall results fo^ ♦actual questions are nonsignificant but with a trend 
toward posttive relationships 
with a trend toward positive relationships 

Overall, the four studies m the intermediate grades are more difficult to 
interpret than those m the primary grades because of greater differences m 
coding and statistical procedures As m the primary grades, there is a trend 
for factual questions to be positively related to achievement gam. and for 
open questions which concern student personal experiences to be nega* 
tively related in addition, one study yielded a significant positive relation* 
ship between interpretative * questions and student achievement, but these 
results were not replicated m the other three studies, indeed, m another 
study, there was a moderate number of divergent to convergent questions 
which were significant * Of the remaining two studies, one yielded all non- 
significant results. ' and one found a negative trend between questions 
regarding student interests and opinions and student achievement.* This 
last finding appears similar to that obtained m two of the studies conducted 
m the primary grades " 

In two studies m junior high school grades, both m science, each used a 
ratio of higher level to lower level questions in one. a moderate ratio was 
most functional, and this finding appears similar to that obtained m inter* 
mediate grade vocabulary However, both sets of results must be seen as 
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tentative because students were the statistical unit, in the other study, a 
high ratio was most functional, but this finding was limited to the high 
achieving Children in the sample. The high ratio was also most functional in 
the Single senior high study, one conducted within social sciences. 

The final study, in college social science, replicates results which ap- 
peared in studies in the primary and intermediate grades.^' Interpretation 
questions were positively related to achievement; opimon questions were 
negatively related. 

Summary of Correlational Studlat on Teacher Questlone 

The conclusions listed below are tentative. They are based on 16 diverse 
studies, some of which do not have the best design characteristics. 

1. The frequency of factual questions was either positively related or 
unrelated to achievement gain. 

2. The frequency of higher level ' questions, as a whole, was usually 
unrelated to student gain. This finding held regardless of the outcome 
measure 

3. In two studies which used ratios, a moderate ratio of high level to 
factual questions was most functional. However, in two other studies 
which also used ratios, it was the high ratio of high level to factual 
questions which was most functional. Because of this confusion, 
there is almost nothing which can be said about the appropriate 
mixture between high level and factual questions. 

4. Questions on personal opinions or personal experiences, or other 
forms of nonacademic, open-ended questions were, when studied, 
consistently negatively related to achievement. This finding was 
obtained m primary grades and m college. 

5. When questions were divided into finer groupings than high level 
and lower level questions, then a count of the frequency of interpre- 
tative questions yielded positive and significant results in two studies. 
However, the single experimental study which investigated inter- 
pretative questions did not obtain significant results. 

Of course, it is possible that the overall lack of results may be due to the 
primitive way of grouping questions. Using such gross categories as high 
level or lower level may not be an appropriate research approach. More 
studies will be necessary before one can comment on this problem. 

Experimental Studies on Teacher Quettlont 

Although correlational studies are good for suggesting experimental 
studies, experimental studies are the clearest way of validating ideas. 

Given the amount of testimony on the importance of higher level ques- 
tions, one would expect that if teachers were to increase their use of evalua- 
tive or synthesis questions, for example, there would be a corresponding 
increase in a class' ability to answer posttest questions involving these types 
of questions. The experimental studies to date, unfortunately, do not support 
such common sense notions. To date, experimental studies on variations 
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in teacher questions have usually found no significant differences. 
To aid in interpretation, it is useful if experimental studies contained: 

a) Groups which were exposed to different questioning experiences 

b) Observational data on whether differences in questioning procedures 
actually occurred 

c) Data on outcomes measured 

d) Use of the class as the unit of statistical analysis 

Some 12 experimental studies of this type were found, ranging across dif- 
ferent subject areas and taking place in grades 1 through 11. in three of 
these studies, teachers were trained to ask higher level questions, observa- 
tional data confirmed that the treatments were implemented, and the out- 
come measures were developed by the investigators, to be sensitive to 
student ability to answer higher level questions.'* In all three, there were no 
significant differences between the scores for the higher cognitive level 
treatment and a control or a factual treatment. In two other studies the same 
design was used, but a standardized achievement test was the outcome 
measure.'^ Again, no significant differences were obtained. Finally, in one 
study, the observational data indicated thai the treatments did not differ and 
there were, understandably, no significant differences on the outcome 
measure. 

Significant results were obtained in two studies. In one. however, the 
teachers were supplied with both special materials and training in asking 
higher level questions. The post test contained items related to the special 
materials, items not covered m the control classes. In two other studies, 
teacher^ were instructed to teach either for recall or for concept mastery, 
and they exhibited behaviors congruent with the objective.'* Pupils in the 
recall treatment performed significantly better on the recall post test; no 
differences were found ii. the concept mastery test. 

Finally, there were two studies in which the teachers followed a script 
which provided 70 percent factual questions and 30 percent higher level 
questions, or vice versa. The post test contained both types of questions. 
In one study, those in the second grade classrooms receiving the 70 percent 
higher level questions did significantly better on the post test: m the other 
study no significant differences were found between fifth grade class- 
rooms.^^ In neither case were separate analyses made for the two types of 
questions. 

Thus only two of the 12 experimental studies showed a significant effect 
for asking more higher cognitive questions^ and neither of those was a clear 
study. In one. the t^Achers operated trom a script, and this study was not 
replicated at a higher grade level. In the other, the instructional material 
was different for the experimental and control classrooms.^^ 

Despite the common sense notions, there is no evidence that asking 
higher levdl questions leads to greater mastery of higher level post test ques* 
tions. Thus, there is little empirical support for the minicourses which 
attempt to train teachers to ask higher level questions. Of course, the 
studies described above can easily be criticized as being inadequate in 
design and conception, but such criticism will not create significant results. 
Perhaps mere increases in higher level questions are not sufficient. Per* 
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haps some strategy is mvolved But the appropriate Strategy needs to be 
researched, not asserted 



Current and Ongoing Studies 



Although the numt)er of empirical studies which have been conducted 
IS relatively small, there are large scale studies currently under way which 
promise to add to our Knowledge The following seven stud;es are currently 
in various Stages of initial work, data collection, or report writing 

Person in Charge 

Fredrick McDonald 
lEducational Testing 
Service) 

Fredrick McDonald 
(Educational Testing 
Service) 

David Berliner. Betty Ward 
(Far West Lat)oratory) 

Jere Brophy 

Robert Soar 

Jane Stailmgs 
Martin Kautman 



Organization 

California Commission on Teaching 
Caiitorma Commission on Teaching (Phase \ ) 
California Commission on Teaching (Phase ID 



Research and Development Center for 
Teacher Education. University of Texas 

Institute for Development of Human 
Research. University of Florida 

Stanford Research institute 

Bureau of Educationally Handicapped 
(Project PRIME) 



In each of these studies, observational data is being or has been col- 
lected on classroom events and student achievement, in addition to the 
above, experimental studies involving teacher training modules are being 
conducted or are about to be conducted by 



Organization 



Person m Charge 



1 Division on Effective Teacher Edu- 
cation. Far West Laboratory tor 
Educational Research and Develop- 
ment 

2 Stanford Center for Research and 
Development m Teaching 



David Berhner. Betty Ward 



N I Gage 



Of course, these are not the only studies bemg Conducted At least five 
other experimental and correlational studies were reported at the 1974 
annual meeting of the American Educational Research Association. 
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Questions for Current and Future Researclwrs 

When one iearns that studies such as those listed above are being con- 
ducted. I would encourage workers m the area of CBTE to ask whether 
many of the skills currently being taught as part of teacher education are 
also being researched m these studies For starters, one might consult the 
above lists of sKitls. particularly those which were labeled as promising 
and those iabeied as currently unstudied in the classroom (A useful task 
for those involved m competency based education would be to suggest how 
competencies such as the above list might be coded using a category or a 
rating instrument ) 

The Need tor Reviews of Research 

The above l»sts of competencies suggests the need for reviews of re- 
search on these competencies There is currently a lack of interpretative 
reviews of research Most reviews, such as those which appear m a dis- 
sertation, are written to prove a point (or satisfy a committee) it is rare to 
find reviews which approach a topic with an open mind in order to explore 
a group of studies and seek out whatever generalizations might exist (and 
be forthright enough to admit when generalizations are not there) 

Such reviews are needed, particularly on topics such as those listed in 
the tentative competency skills, if such reviews were written as a matter of 
course, the next steps for research and practice would be clearer. Inter* 
pretative reviews should be acceptable as doctoral dissertations. Such 
reviews make sense because most people do not conduct research after 
they receive the>r doctorates, but they do have to make decisions as to 
practice For most people, experience m integrating a body of research 
would be more useful for their academic life than would the typical expe- 
rience of conducting a single empirical study 

Thus* because of the need for these reviews and because of the impor- 
tance of experience m reviewing. I would hope that more interpretative 
reviews would appear m colleges engaged m competency based instruction. 

Conducting Research WHhln Teacher Education Rrograms 

Teacher training programs, particularly those which are competency 
based, afford a unique opportunity for conducting research studies on 
teaching behaviors and student outcomes (The term, student outcomes, is 
used to refer to a variety of cognitive, affective, and social outcomes which 
are attributable to schooling ) Half of the work necessary for such studies 
IS already done as part of the normal training. That is. the trainees learn how 
to ask higher level Questions, explain, listen, motivate, probe, and use Stu- 
dent ideas and master the list of skills currently taught as teaching compe- 
tencies in most cases, it is easily demonstrated that trainees change their 
behavior in the course of training it remains only to complete the study 
and determine whether this training also results m superior pupil achieve- 
ment 

Such studies are relatively easy to conduct. As the first step, all trainees 
could teach the same unit to groups of pupils One could use as teaching 
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units tha matenats on why birds smg^^ or on arithmotic.^* The&c mamriats 
(as watt as those used m the other studies summarised under Experimental 
Studies, betow) contain both instructional materials and post tests with both 
factual and comprehension questions. Then, after training, teachers would 
teach the seme unft to other groups of pupils. In both cases, of course. pu» 
pil achievement scores (and other measures of interest) would be obtained. 
The research qu'^stions are: 

a) Does the trainee's behavior change m desire<* directions between 
the first and second trial? 

b) Is pupil achievement superior after the second teacher*s trial? 

At present, there are only a handful of experimental studies which have 
data on both teacher behavior and student achievement, but there are over 
200 institutions that are training preservice teachers in specific skills. If 
each of these institutions conducted only one study wh year, our knowl- 
edge would take a large leap forward. Surely it is not excessive to ask each 
institution to conduct one such study per year, particularly when they are 
already engaged m extensive skill training. 

Designs tor wi>erlm9ntBl studies. The above design is presented below 
under Design One It is Group I that was discussed above. There is. of 
course, the possibility that the experience of teaching the same unit twice 
is helping the teachers more than the training is. To answer this question, 
one might include a second group (Group ID which also teaches the unit 
twice. The research question would be whether pupil achievement on the 
second trial for Group ii will be lower than achievement for pupils taught 
by teachers m Group I. 







Teach 


Training 


Teach 


Design One 


Group 1 
Group 11 


Unit A 
Unit A 


yes 
no 


Unit A 
•Jnit A 


Design Two 


Group Ml 






Unit A straight 

Unit A Sihowing 
results of training 




Group IV 


Unit A 


yes 


Units 

(or Unit A or both) 


Design Three 


Group V 


Unit B 


yes 


Unit A 

(or Unit 6 or both) 




Group VI 


Unit A or 


no 


Unit A or Unit B 



Units 

Design Two is a variation of Design One. in this case the teachers do 
their teaching after training. Each teacher teaches the unit twice— once 
using the skills they learned, and once without the skills. When this was 
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done, ihe lessons were observed to determine whether 
a) The lessons differed m the training variables 

t» The lessons differed m other variables which are currently con- 
sidered important for instruction, and 

c) There were differences m student learnmg and attitudes between 
the two types of lessons 

It IS important that the lessons differ in a) but not on b). The advantage 
of lessons m which each group acts as its own control is that the effects of 
experience can be controlled by random utng which type of lesson is taught 
first 

A third type, Design Three, yields the best information, but can only be 
used when there are at least 50 teachers, in such a caso, the trainees are 
divided into two groups; one group begms by teaching Umt A and the other 
by teaching Unit B. After training, each group teaches the other unit. In 
this way one can obtain data on two units of instruction and thus check 
whether the results are consistent across two instructional units. 

fine could add to Design Three by having trainees teach both Unit A and 
Unit 6 after training. By this procedure one would have a larger sample 
for analysis and could also determine whether previous experience with 
a unit affects student achievement and attitudes if possible, a control 
group. (Group VI) would add to the interpretation of the results 

NumlMr of Skills for an EKporlmontal Study 

Most experimental studies conducted to date test the effects of training 
m a Single skill such as Questioning, use of student ideas, or introducing a 
lesson. But studies need not be limited to single skills, it is possible to have 
preservice (or mservice) teachers receive thtc special training. It is possi* 
ble that significant effects will be obtained more easily when training takes 
place in a number of skills 

Another procedure would be to run correlational type studies to relate 
teaching behavior and student achievement within each of the trials. 



Matortalt for Experimtntal Studlot 

Three sets of materials are useful m conducting experimental studies: 

a) Modules or other instructional materials for training teachers 

b) Short instructional units 

c) Tests to assess student learning 

Training matanals. Teacher training materials are abundantly avail- 
able.'^ The current task would be the validation of existing materials and 
not the creation of still more unvalidated materials. 

Tha ahort instructional units are currently available from the researchers 
who have conducted studies usmg these units These include the two week 
units in social studies and m mathematics used by Flanders,^* the three 
week unit on ecology prepared by the Far West Laboratory,^' the short units 
on mathematics and on social studies developed at Indiana University, 
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and a 30 minute unit on the N^r K.t)tHed Quti.^** 

The proposed studies would occur mure easily if someone assemt>led 
these units and made them available to others. Such a task would be a 
useful contribution to CBTE assessment. 

Popham and his associates have developed a series of 15-minute selec- 
tions and post tests which contain selections dealing in all subject areas. 
Whether IS^mmute units or longer units are the n^ost appropriate vehicles 
for research remains to be determined (in studies to date the results were 
Similar across different lengths of instruction.) Hopefully, a similar kit of 
materials and test instruments could be developed by assembling those 
which have been used in previous studies. 

resfs. There are currently two organizations whr.::h are assembling and 
evaluating the available tests in a variety of cogi. : ^ .^nd affective areas: 
the ERiC Clearinghouse in Tests and Measurement, located at Educational 
Testing Service, and the Center for the Study of Evaluation, located at 
UCLA. Each organization has published a number of evaluative descrip- 
tions of relevant instruments. In addition, handbooks of tests in specific 
areas are starting to appear.'^ Thus, one who is looking for suitable tests 
will find that there are organizations concerned with helping them. 

Teaching Com|Mt«nct«t and Curriculum Packagat 

One area for teacher training and research which is currently neglected 
is the use of curriculum packages. By curriculum packages, i refer to 
packages such as Biological Sciences Curriculum Study. Elementary School 
Science. Taba Social Studies Curriculum, and individually Prescribed In- 
struction. 

These packages and others of this type include something more than the 
usual curriculum guide or the usual textbook. (Of course there are many 
which are the usual curriculum guide and admonition to "do good.'* but 
there are enough packages such as the above to merit discussion here.) 

The curriculum materials packages offer at lear* !hree advantages: a) the 
materials, instructional procedures, and sequencing were developed by a 
team of specialists m content and instructlori; b) tne products have been 
revised and refined on the basis of internal evaluation and try-out feedback 
obtained from 20 to 200 classes; and c) a good deal of summative data is 
being generated which will help m conducting research on the relative 
effectiveness of the curriculum programs for different educational out- 
comes 

Although there are a large number of these programs, they can be 
grouped into three or four approaches. One grouping is the highly struc- 
tured approach exemplified by a program such as Individually Prescribed 
Instruction or the DiSTAR package. Another grouping might be structured 
discovery as illustrated by the Biological Sciences Curriculum Study pro- 
grams or by Harvard Project Physics. A third grouping is tentatively called 
"teacher directed inquiry * and is illustrated by programs such as the Taba 
Social Studies program or the Holt Social Studies program. A fourth group- 
ing is that of more informal, open approaches such as the Elementary 
School Science program or the model developed by the Bank Street College 
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of Education 

1 would propose that in addition to training teachers in the lists of com- 
petencies developed by a committee, we a/so help teachers study these 
different programs and then tram them in the competencies necessary to 
implement these different types of programs in other words, why not bring 
curriculum and instruction together, instead of leaving curriculum as a 
junior partner to teacher training in teaching skills. 

Curriculum Materials Packages Within Preservice Training 

HOW might curriculum packages be used m preservice education? in 
elementary mathematics education, for example, we don't know which 
Curriculum product, it any, the teachers will be using throughout their 
careers. However, as illustrated above, one can group the existing pack- 
ages and the ones being developed into two. tnree, or four types and train 
teachers m the skills necessary to use these types of materials. Preservice 
teacher training can be conceived of as training teachers in the skills 
required by the types of educational products that are expected to be in 
use during the early part of their careers. Preservice teachers would learn 
the skills by using and studying the different typea nf curriculum packages 
In elementary mathematics, for example, teachers would learn skiMs of 
informal teaching by attempting to implement the mathematics portion of the 
Bank Street or the EDC models: they would learn skills of structured math- 
ematics teaching by learning to implement programs such as the 
Englemann-Becker or the Karnes, and they would learn skills of ^modern 
math" by working through sections of the Qreoter Cleveland Math Pro- 
grams or the Madison Math Prooram. 



Curriculum Programs and Accountability 

if curriculum materials packages are being used (and if development of 
new materials and programs is assigned to appropriate agencies), then the 
responsibilities are reallocated among the various groups. Teachers ar:s 
responsible for implementing the program according to the instructions of 
the developers. If teachers decide to change the instructions, then those 
teachers become accountable for the success or failure of the students to 
learn. The developers are responsible for determining the most appropriate 
instructional procedures for the students whom the teachers will have, and 
if the teacher follows the instructions, then the developers are responsible 
for the outcomes The developers and the teacher educators are responsible 
for prnwiding the io^cher training; the administrators are responsible for 
making the training available and for providing sufficient teacher training; 
and the public is responsible for providing the funds for the materials and 
the training. If the program includes parents working with the students, 
then the parents are responsible for their contribution. 

In other words, rather than each group trying to fix the blame on the 
other, in this situation each group is responsible for facilitating the work of 
the other. 
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This proDosed realignment of responsibilities seems analogous to that of 
a physician who is responsible for using the best machinery and available 
knowledge in his diagnosis and treatment, but is not responsible for curing 
heart disease and cancer. The responsibilities for finding cures and drugs 
belong to special and appropriate research organizations. Of course we 
want people cured of cancer; but will it be done by setiing down a list of 
-performance competencies ' for physicians? Of course we want more hu- 
man interaction; but will it be done by setting down a list of -performance 
competencies ' for teachers? 

Netfl«d RMaarch on Curriculum Packages 
and Taachiitg Compatanctat 

The above remarks on the usefulness of curriculum packages need to be 
tested One needs to research whether the teaching skills which are part 
0' -i^e curriculum packeges are indeed related to student achievement. At 
present, there have been only a few studies in this area. In one set of cor- 
relational and experimental studies, significant results were obtained on 
ihe importance of teacher correction procedures in a highly structured 
curriculum program. In other studies using other variables, the results 
have not been c'.earcut.^^ 

Summary 

The major purpose of this paper is to call for more research on teacher 
competencies and more reviews on the results of previous studies. Because 
of the Importance of this research. Institutions involved in teacher compe- 
tency training are urged to conduct one study and one review of research 
per year, tf such work is initiated, then within two years we will nave a 
cleare*- knowledge base from which to proceed in training and research. 
Suggestions for conducting such research are provided within the paper. 
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11 If the Program is 
1 1 Competency Based, 

How Come the 

Evaluation is 

Costing so Much? 



Michael Scriven 
Introduction 

A competency based program is presumably thoroughly developed by the 
time It is in full-scale operation, which presumably means that it gets its 
graduates up to the designed level of quality and quantity of achievements. 
(We pass lightly over the fact that it may do this by lowering the design 
level in the light of realistic field trials. />iere at last, one feels— or is it just 
that one hopes?— is a program that knows what it does and says what it 
knows. Here we have laid to rest the misleading advertising of inflated 
phrases and vague promises that pervade traditional education catalogs. 

Apart from the moral and scientific charms of the competency based 
approach long extolled in prose and poetry (somehow the poetry seems to 
creep back into the catalogs), there does seem to be at least one obvijus 
pecuniary advantage about this innovation, namely the Aiimination of any 
need for fancy evaluation Competency based education, the honest f'^llow. 
wears his— or is it her- crcaentials on its sleeve. They need no embellish- 
ment. 

I wish I could add to your enjoyment of this evening's banquet b/ en- 
couraging you to believe ih«ii if le competency based path is indeed rewarded 
in this way. but I propose instead to contribute to indigestion by stressing 
that the direct cost and quality of evaluation in competency based programs 
that I know about are not even in the ballpark for adequacy. Not. I hasten 
to add. that this puts it in a poor position compared to most educational 
programs. While it is true that the competency based approach offers the 
possibility of a modest saving in one dimension of evaluation, this saving is 
usually not realized or not realized fully; and competency based evaluation 
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designs too often leave unanswered several fundamental questions about 
th€/ merit of the program. My chief interest this evening is to outline the 
basic features of a satisfactory evaluation design. 

To ensure a sweet aftertaste to this rather bitter post*coftee cordial, I 
shall add a word or two about the costs of evaluation, directed to the thesis 
that the median net cost, to the appropriate agency, of a thorough evalua- 
tion. IS negative. 

The Main Problem 

The main proolem with the competency based movement*, as I see it 
today. IS not— >as the sideline cynics are always saying about innovations— 
that It does the same old things and just gives them a different name. There 
really i$ a different orientation and a different methodology, as well as a 
diffe('*^nt vocabulary. The point can be quite well illustrated by reading the 
program for this meeting. It evidences no mere rechristening activity, but 
an effort by many participants to focus and explain their aims more clearly: 
an effort that definitely affects the content of at least some of the presenta- 
ttons. 

The mam problem, however, is still something pretty simple. It is the 
problem whether the totai effect of the new orientation and methodology is 
worth the very considerable effort it has involved, especially in light of the 
dubious juGtiftcation of many of the competencies aimed at. The aim may 
wen have been successful, but U s not at all clear that it was sensible. 

The analogy with the behavioral objectives and criterion-referenced 
testing movements naturally springs to mind. As I travel around the major 
metropolitan school districts talking to school principals and district or 
state staff. I beiieve 1 have noticed a steady deterioration in the interest 
m what I call the oredentialing of objectives or criteria at the expense of a 
pro forma interest in behavioral objectives or criteria as such, or perhaps an 
interest m them for what they contribute to conceptualizing the educational 
activity, whether planning, performing, or reporting it, (e.g.. also program- 
ming of texts.) 

Precisely the same problem-*as is no doubt obvious to most of you — 
faces the competency based movement It began with a no-nonsense orien- 
tattrn towards pragmatic teaching and evaluation, with emphasis on pay-off 
skins. But pretty soon we begin to f;nd that the concept of a competency 
floits froe of the original identification as an indubitable necessity for satis- 
factory job performance, and is taken to include all sorts of juggling acts 
that iomecr.e has taken a fancy to. These are process competencies, 
and efforts to disseminate them in the absence of proof that they're impor- 
tant fo' pay-off performance represent exactly the kind of faddist, ill- 
grounded approach which the competency based orientation was instigated 
to avoid. Only an uncompromising commitment to systematic and compre- 
hensive evaluation of each program can avoid this kind of rot from under- 
mining what could provide the foundations of a better approach to teacher 
education and to many other kinds of education. Tm delighted to see 
symptoms of concern with this point in some speaker summaries, e.g., N. 
L. Gdge*s reference to the need for first establishing desirability in teacher 
behavior. It's equally conspicuous by its absence in many other summaries. 
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Perhaps that's just because they are summaries; we shall all find out soon. 

Th# Basic Rtqubr*montt for Good Evaluation 

Let me try to provide. In checklist form and as concisely as possible, an 
outline of the points that must be covered in the evaluation of a competency 
based teacher education program. They derive from a general checklist, 
which was originally developed for the evaluation of educational products, 
but which has since been expanded to apply to programs and projects (as 
well as to evaluators and proposals, etc.). This presentation will avoid the 
illustrative but slightly irresponsible anecdotes used in the Houston talk, 
where we were facing all the problems of after dinner speeches, and it will 
omit the details of rating scales, etc.. which are required in order to 
implement a checklist evaluation in practice. The intention is simply to play 
out the many dimensions which have to be explored in the course of evalua- 
tion, and which are not met by the mere waving of the magic wand on which 
are inscribed the words ' competency based.** 

Need 

The first question, of course, is the one that I mentioned a little earlier — 
the question of the justification for this particular competency, set of 
competencies, programs committed to teaching us this set of competen- 
cies, etc. (For convenience, ril focus on a component of a competency 
based teacher education project, which is devoted to teaching a particular 
competency.) The problem of credentialing any successful training pro- 
gram is essentially the problem of hooking up its products to a needs 
assessment. Can we show that the need in question is a rea/ need? Needs 
assessments are typically done in two completely spurious ways; either by 
using standardized tests and assuming that short falls on these tests repre- 
sent need, or by doing a survey of the population that will be affected (or 
that is thought to be the population that will be affected) by the training 
program. The deficiencies with each should be obvious, although both are 
provided as examples in current textbooks devoted in whole or in part to 
program evaluation. There is no need in the world to be performing at norm 
level on any standardized test. For that matter, there isn*t any need to be 
performing at norm level on any criterion-referenced test, unless the 
criterion can be demonstrated to concern, e.g.. survival or ability to cope 
with further experiences that will be leading one towards an adequate col- 
lection of survival skills, etc. Given the differentials in rates and direc- 
tions of maturation, it really wouldn't make much sense to say that some- 
body had a need to perform at norm level (mastery level) even on a criterion- 
referenced survival-connected curriculum excepf at the exit test. 

The deficiencies in the usual survey approach are equally serious. People 
are very poor judges of what they really need (as opposed to what they 
want) where the issues they are passing judgment on are either technical 
or complex or emotional or novel. Since most of the areas where the sur- 
veys are taken concern issues that meet not just one but all of these condi- 
tions, they are staggeringly unreliable guides to need. Tm not suggesting 
for one moment that there is some omniscient entity around who can sim- 
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ply and reliably identify the **true needs** of populations of people. People 
may well pe the Pest judge; Put that doesn t me%in that theyVe the Pest 
judge in the short run. when uninformed about the issue, and when torn by 
conflicting concerns over it. There are. of course, areas in teacher educa* 
tion where these problems are not serious: but if you start looking into the 
question of values clarification m social studies curricula, citizenship edu- 
cation, sex education, propaganda analysis, ecology, race relations, career 
choice, sex role, and a dozen other examples from the everyday activities 
of teachers in the school classrooms of 1974. then you can readily see that 
the survey approach to needs assessment is likely to be unreliable, rm not 
here to offer solutions— a solution to this problem requires some specifica* 
tion of the exact version of the problem that is facing one at the particular 
moment—but only to indicate the nature and dimensions of the problem, 
rii leave this point with the single thought that needs assessments of suit- 
able degrees of reliability can indeed usually be done, but it takes a tot of 
work and thought to do it. 

Market 

What is needed is often unmarketable, and what is marketable is often 
not needed; once the distinction has been recognized, it s clear that we 
have to investigate market separately from need, or else we will be left with 
a training program for which the development money will have been wasted, 
since it cannot be disseminated to any significant number of the centers. 
NlE is in the process of rethinking its commitments in the direction of dis* 
semination. This is a most welcome move, since there has been a long 
tradition of assuming that dissemination will take care of itself, an assump* 
tion which shows an optimistic rather than a realistic nature. 

Performance Data—True PleM Trials? 

Do we have a training program that really can deliver the promised com* 
potencies? Often we have one that has operated very successfully in an 
' intensive care** situation, guided by the inventor's hands and enthusiasm, 
but never tested out in the field, where it s got to sink or swim by itself. 
Where there have been field trials, they*ve often been semi-subsidized, with 
materials or assistance provided free of charge and that welcome little 
feeling of being part of the in-group being supplied without any charge at 
all. As long as you can coast along on the Hawthorne effect, that will work 
splendidly: but it*s a disservice to everybody ;o suppose that you can dis* 
regard the extent to which you owe your success to that handy little booster. 

Performance Data~True Conaumeri? 

Who have we looked at as the impacted population for our little educa-^ 
ttonal package? Obviously, its primary impact (chronologically, at least) is 
on the teacher trainees who are going through the programs. A good evalua- 
tion doesn t rest on its laurels after looking at the first contact. After all, the 
justification of this little module is meant to lie. not in its beneficial effects 
on teachers, but in their beneficial effects on their students. So we normally 
would need to look into the question of whether we have data on the even- 
tual student effects. The widening ripples do not cease at that point, how- 
ever; do we have data on impact on future employers? What about impact 
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on taxpayers'^ What about impact on other learning activities of the first 
level of students, i.e.. the teacher trainees? What ak)out special demands 
placed on the trainers themsetves--have we investigated them as an im* 
pacted audience? The real consumers are a much less elite group than 
most evaluations suggest And the news from the distant shores to which 
the effects finally spread is often not as good as the news from the points 
close to the point of first impact. 

Pert ormanct Data-^CrWcal Competttort 

Useful evaluation is always— implicitly or explicitly-^comparative. The 
real question for the person who is thinking about adopting or purchasing 
a competency based program is whether it is better than alternatives, not 
whether it is better than nothing at all. Hence, the evaiuator sets up. or 
explores ERiC. for evidence about critical competitors. The choice is not 
between nothing at all. but between actual competing products or programs. 

So. the Choice is between a competency based program and an existing 
program, or between the competency based program and an alternative 
innovative program, e.g.. apprenticeship systems with on-site workshops, 
etc. 

Now, in the vangu&rd of new movements, there is always considerable 
resistance to having a new movement compared to the old one. The avant 
garde of the new curricula projects of the last decade were always talking 
about how inappropriate it was to compare them with the old ones. The 
legitimate part of this point was that it would have been unfair to compare 
them with the old ones, using only the old ones' criteria for the standards 
of comparison. However. it*s not at all inappropriate to compare the new 
and the old on standards that embrace the best points of each. It is a matter 
of considerable significance to discover that Japanese children outperform 
their American age-peers, both on the Japanese curriculum tests and on the 
American curriculum tests. Whatever the explanation of this is, and it lsn*t 
necessarily a sign of the failure of the American school, the evaiuator is 
making an extremely useful contribution when he or she identifies this dif- 
ference. Now you can bet your life that competency based approaches 
a/ways have been compared to traditional approaches on at least one di- 
mension- -namely, cost. And the fact is that it's compared on a great many 
other dimensions too: the extent of disruption, opposition by staff, etc.. etc. 
It s a nice, comfortable little insulation move to suggest that the new is 
only to be evaluated on its own terms. But it's totally unrealistic, and it*$ 
illegitimate, it took many years for economists to discover that— to their 
embarrassment— a competitive market for refrigerators embraced not only 
refrigerators, but dishwashers and electric stoves. That is. the rational 
consumer quite properly (and not irrationally) weighs the purchase of a new 
appliance against the purchase of other possible new appliances. Well, so 
It IS with the competency based approach. It has to earn its spurs in the 
free-for-all of the educational marketplace, not in some sequestered cloister. 

Performance Data— Long-Term 

We're very inclined to suppos'^ that the termination of treatment is the 
appropriate time to measure the effects of treatment; and we thereby con- 
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fust logistical simplicity with methodological propriety, it's exceetf/ng/y 
inconvenient to have to segregate funds for use m long-term funds; but it's 
eKC99<i9ngly unreliable to act as if somehow the results of such a follow-up 
will automatically be the same as the results of an evaluation at the termina- 
tion of treatment 

Perfof menct Dau — SMe Erteclt 

Has there been a systematic, scientific, and sophisticated search for 
Side effects'' it has to be borne in mind that from the point of view of the 
eva/uafor. the goals of the program are really not a matter of great impor* 
tance They're certainly interesting for the archivist, for the developer, 
and for the funding agency But what interests an evaluator is what the 
program actually did. judged by its congruence with the needs of the 
impacted population, it doesn't really matter much whether what it did was 
what It meant to do. or whether it failed to do what it meant to do, but did 
something else instead, etc.. etc it's up to the evaluator to find out exactly 
What It did. and. of course, this means a great deal of fishing in murky 
waters where no guidance is provided by reference to goals statements. 
The search is not altogether without its own attractions: the murky waters 
of goal statements, frequently inconsistent or incompatible with the goal 
Statements of others associated with the project, or out of date, or based 
on factually false assumptions, etc.. are by no means obviously more attrac- 
tive than those wherein side effects lurk. 

Performance Dala—ProeeM 

It s quite wrong to use process as a reliable guide to outcome, but that 
doesn't mean that there aren't legitimate reasons for inspecting process in 
the course of evaluation. Four reasons; are mentioned here: 

a) The search tor injustice. Whatever the outcomes, there are certain 
types of procedure that are inappropriate for moral reasons, and 
inspection of process must be made to see whether excess cruelty, 
inequitable methods of grading, etc.. are important. 

b) The search for joy. The second question is whether there is pleasure 
in the learning experience. There are other reasons for trying to make 
learning enjoyable, namely, that to do so is likely to increase the 
probability of learning taking place. But if all else fails, if learnmg 
gams are negligible and if the cost is comparable, then we might as 
well take account of the question of whether the babies enjoy baby- 
sitting of type 1 more than they do type 2. I 'd say that joy is a second- 
ary end of education (because if it's joy you want, then for most 
people education has some overwhelmingly critical competitors), but 
that doesn t mean it shouldn't be an aiin. 

c) Characterization of the process. When the evaluation is done. some> 
body is going to have to label the package that was evaluated. Of 
course, it had a label when it arrived at the evaluator s doorstep, but 
that label is as often as not extremely misleading: and one of the 
tasks of the evaluator is to decide whether this indeed deserves the 
title of competency based" training or not. 

d) Causation Inspection of process often provides valuable clues to 
determine the responsible agent for any changes that are observed. 
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In particular, it may be necessary for you to inspect process in order 
to settle the question of whether the alleoed cause was the real cause 
of the changes, in situations where control groups and baseline data 
are impossible. 

Peftormanee Data— CautatloA 

Expanding on the last thought of the previous subsection, we have to 
face the fact that almost every evaluation constitutes an investigation of a 
causal claim, i.e., the claim that such-and-such a procedure had such-and- 
such (allegedly beneficial) effects. This is one of the reasons why evaluation 
research is by no means an easy type of research. Quite often in teacher 
education, the amount of learning that takes place from peers is distinctly 
comparable to that which occurs from the training process itself; and the 
same applies to 'natural*' learning from field experiences as opposed to 
the contribution of the supervisor. 

Pefformance Data--8latlttlcal Significance 

It IS scarcely necessary to mention that evaluation frequently requires 
some rather elaborate statistical investigations, in order to determine the 
true significance of an apparent effect; or indeed in order to identify any 
effect whatsoever. 

Educational Significance 

When we are m possession of reports on the first ten checkpoints, we 
are in a position to tie them together and come up with a conclusion about 
educational significance. All too often, educational significance is identified 
with statistical significance of the difference between the treatment group 
and thi^ control group. But of course that difference may be due to the use 
of a very large number of subjects (which magnifies trivial effects), to in- 
valid tests, or to a dozen other causes, none of which represent any con- 
tribution to education. Checkpoint 11 is thus a summing-up checkpoint, 
where the results of detailed item analysis may come in, where investiga- 
tions of instrument reliability may come in, and where It is frequently neces* 
sary to refer back to one*s needs assessment data, in the light of one*s 
refined characterization of the product, to see whether in fact the product, 
as it turns out to be, really meets the need that it was supposed to meet. 
Where the product-*-as in our case— is a competency based training 
module, this kind of worry translates itself into worries about whether one 
can view the little skill that was acquired as generaiizabie to the many other 
Similar skills that will be needed in the classroom. 

Cost Eff ecthrenesi 

We now have to Iook into questions of cost, and what a very complicated 
business that is, indeed. The use of a C.P.A. Is not enough; the use of an 
economist Is not enough: we usually have to use an economist with special 
expertise in the field of school economics: and a CP A. with special exper- 
tise in the various types of bookkeeping used by schools at primary, second- 
ary, and tertiary levels. 

When we have adequate data on the real, the opportunity, the direct and 
the indirect, the discounted cash flow, and other costs and analyses, then 
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Me can combine them with the results of checkpoint 1 1 to yield either a 
cost^tenefit analysis (at least) or a cost*effectivenes:i analysis (in most 
cases where we find reasonably simple dimensioning) is possible 

Contlmiing Support 

This is a funny little item, as ttefits its number and its location on the 
Checklist But It s something of the sting in the tail of the checklist, too. 
it concerns the question of avoiding orphans; when we start looking at 
orphans in the educational marketplace, we are much struck by the fact 
that the little entities that everybody wanted to be heir apparents to on one 
day are often entirely orphaned the next. Faddism is the great enemy of 
educational progress; in order to avoid faddism. we need, in the course of 
a serious evaluation, to look at the question of sustained support and im» 
provement in a given educational program, project, or product. Will Sony 
stop manufacturing video cassette recorders in the near future, just after 
we ve purchased a set of them for the school? Will the behavioral objectives 
movement indeed be steadily amplifying the libraries of instructional ob* 
jectives after we convert to a commitment to behavioral objectives? Will 
there be programmed texts that the students can use once theyVe through 
this set of programmed texts, or will they be programmed orphans? Will 
the competency based movement follow through? The evaluator has to 
investigate such questions. 

TiM Sp«clf ic Probltm of TtacNir Evaluation 

I will simply mention the fact that, apart from the problem of evaluating 
competency based teacner education, there is also the problem of 
evaluating the products of competency based teacher education programs, 
namely, competency trained teachers. Here we run into a number of other 
quite different problems for the evaluator. How do you get from a conclusion 
about the merit for the impacted population of having compet'^ncy based 
teachers to the merit of an individual teacher? There is no easy path. Even 
such simple questions as how one gets from competency-mastery judg* 
ments (which one makes in the course of supervising the trainees) to 
competency-exerc/se conclusions (which is what one needs in order to be 
sure that his competencies will be used m the classroom)? James Popham 
has clearly stressed the important distinction between competency based 
approaches to program evaluation and competency based approaches to 
individual teacher evaluation; given the present instability of data on com- 
potency of teachers as individuals, we by no means have to throw up our 
hands about the possibility of evaluating training programs using compe- 
tency based tests. Again, there is the question of whether we are going to 
evaluate teachers for minimum competency, or for purposes of developing 
a competency profile on which one could rely for. e.g.. placing a specialist. 
The former is in much better shape, as the evaluation art stands, than the 
latter 

In short, while the competency based approach offers us considerable 
opportunities for improving the present state of teacher evaluation, this 
mainly shows that the present state of teacher evaluation is in very bad 
shape, not that the competency based approach is without its own problems. 
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CotMrM Evaluation 

I promised to add a tittle touch of good news to the end of this list of 
worries, and I wish there was time to develop the good news at comparable 
length! But I can at least explain the general thesis that I want to propose, 
even if i can't get mto details about the exact reasons for believing it to be 
true. 

The general thesis is that the costing oi evaluation Itself is usually incor- 
rectly done. It is usually seen as a cost on the books of either an agency or 
a project or perhaps a school district; the theory there reflects the fact 
that somebody has to make out a check to pay the evaluator. What doesn*t 
show up there is the hidden side of the ledger, namely, the returns from the 
evaluation. Exactly the same point can be made about a fire department. 
!f you ask the mayor of a city what the cost of the fire department is, then 
he or she is likely to reply by mentioning the figure that represents the 
amount of money paid over by the City Treasurer into the fire Department 
account each year. Of course, the fire department isn*t lust a cost. The 
department's existence is what keeps insurance rates down in the city. 
The department s existence is what keeps the death rate and property 
damage rate down m the city. The returns from those activities of the 
d<»partment have to be taken into account in a correct set of social books. 
The same is true of evaluation Evaluation is not a tack^on luxury, stuffed 
down our throats by external agencies, it is what it takes to determine the 
quality of one s own work Without effective evaluation, there can be no 
knowledge that what has been done has been worthwhile. //. therefore, 
one s commitment is to worthwhile activity, then one is necessarily com- 
mitted to substantial evaluation. Now, if we start drawing up a set of social 
books, then we have to recognize that evaluation is frequently responsible 
for terminating projects that are not worthwhile; and it is sometimes respon- 
sible for increasing the amounts of resources that are put into other projects 
of exceptional promise, thereby getting them beyond the threshold of very 
large scale payoffs. Evaluation, in short, should not represent a net cost to 
a community, such as the teacher educators. I won t say that bad evalua- 
tion doesn't cost more than it should; by my standards, that's an excellent 
ground for saying that it*s bad, since it fails to outperform a critical competi- 
tor. I only say that a good evaluation is not only a good investment but the 
only means whereby you can recognize whether your investment is good or 
not. So. if I presented a rather depressing list of activities^ that are 
involved m satisfactory evaluation and that are not taken care of by the 
move :o compeicyncy based approaches. I have at least p^^ovided some kind 
of a sugar coating for the pill that I am suggesting one has to swallow! 

Conclutlon 

Competency based evaluation is a means to the ends of education. 
Whether it becorrtes one more semantic tombstone in the cemetery of odu* 
cational fads depends entirely on two issues: first, whether we can Keep our 
eye on quality and quality control, and meet the standards of quality, not 
mere popularity; second, whether we can act like lational users or con- 
sumers and adopt on the basis of merit. 
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Competency based teacher education is an operational reality at rela> 
tively few institutions. At most other teacher education institutions, the 
competency based movement is an emergent phenomenon that is proceed- 
ing at various rates. Evaluation of such emergent competency based pro- 
grams must, of course, fulfill all of the usual functions of program evaluation: 
It must provide objective, impartial measurement of the effects of the 
program. In addition, program evaluation should form the core of what is 
in effect a management information system designed to provide program 
decision maker«> with accurate, objective, timely information concerning 
the relative effectiveness of all elements of the system. Such evaluative 
feedback is essential for effective management: without it. crucial deci- 
sions often will be made on the basis of hunch or expediency. 

Ideal program evaluation as seen by the Task Force members measures 
program success against the 13 criteria discussed by Scriven during this 
conference These cnterta rest on what is essentially a systems approach 
to evaluation— an approach which considers not only the extent to which 
the organization is successful m reaching its stated goats, but which also 
considers things which are in a sense subordinate goals (for example, the 
level of morale among people involved in the program). Program success is 
seen as a complex, multidimensional concept. The evaluator should study 
not only the degree to which the program achieves its goal of improving 
education by improving the preparation of educational personnel, but also 
the processes through which the organization functions. 

At the most basic level, the keystone of performance based education 
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programs is the willmqnebh to be heiU accountable for achieving stated 
program objectives through the use of training and assessment Procedures 
which are also open for public inspection The openness to summative 
evaluation implied by this may be at once performance based education's 
greatest strength and its greatest weakness The strength lies m the 
Willingness to be evaluated on the basis of publicly stated objectives, but 
It IS also a weakness m that premature summative evaluation could result 
in the dismissal of performance based education as a viable route to the 
training of better teachers before it has receivet* > fair test. The Task Force 
believes that, although performance based teacher education must ultimately 
stand or fall on the basis of summative evaluation which compares teachers 
trained by performance based programs with those who graduated from 
other kinds of programs, such summative evaluation should not and cannot 
be undertaken until performance based programs have had the opportunity 
to mature. 

This does not. however, mean that attempts at program evaluation should 
be postponed until PBTE programs have achieved some optimum stage of 
development at some time in the indeterminate future. On the contrary, 
there is a real and present need for systematic efforts at formative evalua- 
tion—evaluation which collects data on the success and efficiency of many 
aspects of program operation, data which are then fed back into the pro- 
gram Itself in such a way as to allow program decision makers to optimally 
shape the course of the program. It is the position of the Task Force that 
performance based teacher education programs should be judged today, not 
so much by some absolute standard of accomplishment (summative evalua- 
tion), as by the relationship between their formative evaluation systems and 
their decision making structures. Given the state Of the art. the ideal evalua- 
tion system ts one which constantly provides program management with 
objective, reliable data on all aspects of program operation: conversely, 
the ideal program manager is one who bases his decisions on data obtained 
from just such a formative evaluation system. 

Such a formative evaluation system should collect data on several as- 
pects of program operation. Of central importance »s the evaluation of 
program attainments (e.g.. How satisfactory are the lists of competencies 
developed? Have competencies and training procedures been validated 
against a student achievement criterion? Can teachers graduating from the 
program actually perform the competencies which were specified as objec- 
tives'^ Does the classroom behavior of program graduates differ meaning- 
fully from that exhibited by graduates from other programs?). But program 
evaluation for today's world must also include such process variables as the 
adequacy of the management system which supports the program, the 
'ocus of program decision making, the flow of communications within the 
program, and other factors which together ultimately determine the success 
or failure of the program 

Despite the importance of process variables in program evaluation, 
however, the crucial question for research remains the selection o* adequate 
criteria of program success. This question is not as simple as it might ap- 
pear at first glance, for the success of a teacher education program can be 
judged at many different levels. We might, for example, consider a program 
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successful Only if the students of teachers graduated from the program 
achieved specific levels of academic success. At the other extreme, a 
program might claim success on the basis of data indicating that its grad- 
uates understand the principles of effective teaching and have a good basic 
grasp of their own subject areas An effective formative evaluation strategy, 
however, samples program success on as many levels as possible. This 
approach permits the evaiuator to be highly specific m the delineation of the 
program's strengths and weaknesses. 

The Task Force believes that the criterion levels defined by Turner^ pro- 
vide a valuable heuristic device for this discussion. We basically contend 
that the appropriate level for the evaluation of a competency based teacher 
education program should be Criterion Level Three; a program should be 
considered successful to the extent to which its graduates exhibit the com- 
petencies for which they were trained. 

While i.evel Three should be the central focus of an evaluative strategy, 
other levels should also be considered, individual instructional modules, 
for example, should be evaluated at Criterion Levels Four through Six; 
obviously, trainees should be able to demonstrate understanding of specific 
competencies, demonstrate the performance of these competencies, and 
use them in front of live students, in a sense, these lower criterion levels 
might be considered necessary but not sufficient conditions for effective 
■reaching; a teacher who does not understand and cannot demonstrate a 
competency m a limited situation can hardly be expected to use it effectively 
in a real classroom situation. 

Finally, while it would be unreasonable to hold teacher education pro* 
grams accountable fur the academic achievement of the students of teach- 
ers who graduated from the program, a comprehensive formative evaluation 
strategy should also include research designed to assess the relationship 
between the competencies covered by the program and student achieve- 
ment Program decision makers and evaluators must know how well the 
teachers whom they trained are doing with their own students, for without 
this information they cannot adequately evaluate the competencies which 
are the basis of their programs They also need to know the extent to which 
teachers trained by their program demonstrate under actual classroom 
conditions the competencies which they acquired in the program. 

What the Task Force is suggesting, then, is that program evaluation for 
today's performance based teacher education programs should possess 
three characteristics: (a) it should be formative rather than summative; 
that is. it should be used not to judge the ultimate success of the program 
against some absolute criterion, but rather to help program managers shape 
the program toward ultimate success; (b) program evaluation should have 
a systems orientation rather than a goal orientation ^evaluation should not 
end with the determination of whether or not program objectives have been 
achieved, but should also address itself to the processes by which the sys- 
tem functions: and (c) program evaluation should be supplemented by 
program validation — in addition to determining the extent to which the sys- 
tem reaches *ts goals, a research effort should be aimed at determining 
the validity of those goals m terms of the relationship between the com- 
petencies developed by the program and student growth. 
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FOOTNOTE 

' ^. L. Turner. Levels of Cntena.' «n B Rosnor. ed . Th9 Pomw of Comp^t^ncy Based 
feec/ter Ctfucanon (Boston. Mass Aiiyn and Bacon. 1972) 
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Criteria for 
Describing and 
Assessing 
Competency Based 
Programs 



W. Robert Houston. J« Bnico Burko, 
Ch«rtM E* Johnson, John H* Hanson 

During the past few years competency based education programs and 
projects have proliferated extensively. Some closely reflect the criteria set 
forth by Elam.' Other programs claiming to be CBE appear to be only slight 
modifications of more conventional approaches. Surveys^ of CBE practices 
reflect considerable activity, but the quality appears to vary greatly. Some 
have Simply translated their old programs into the "form" of CBE, while 
others have diligently applied CBE principles. 

But both claim to be CBE. Attempting to describe or compare results of 
such programs is an almost impossible task. 

Beginning in 1973. the Consortium of CBE Centers^ began a project to 
describe the various dimensions of CBE as reflected in operating programs. 
Such a tool could provide the basis for activities such as: 

( 1 ) Surveys of CBE activity 

(2) Self-assessmen^ of intent and progress by CBE programs 

(3) Planning a document to be used by professional preparation 
programs 

(4) Discussion device for considering the function and value of various 
criteria 

(5) Research in institutional change, programmatic strategies, and 
organizationaJ constructs. 

The following criteria serve as the basis for -his effort. While still regarded 
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as a working list, it represents the third major revision and considerable 
debate over the past year by a wide range of persons. In its final form, to be 
published by the Multi-State Consortium in the fall, 1974, each criterion will 
be supported by a set of indicators and program descriptors. 

The purpose of this i* *o provide another tool in the improvement of 
professional education pru^prams. Feedback from readers relative to these 
criteria and to the finished document will be appreciated. 

Criteria for Assessfnfl the Degree to Which 
Professional Preparation Programs 
Are Competency Based 

Competency Specifications 

1. Competency statements are specified and revised based upon an 
analysis of job definition and a theoretical formulation of professional 
responsibilities. 

2. Competency statements describe outcomes expected from the per- 
formance of profession-related functions, or those knowledges, skills, 
and attitude<i thought to be essential to the performance of those 
functions. 

3. Competency statements facilitate criterion-referenced assessment. 

4. Competencies are treated as tentative predictors of professional ef- 
fectiveness, and subjected to continual validation procedures. 

5. Competencies are specified and made public prior to instruction. 

6. Learners completing the CBE program demonstrate a wide range of 
competency profiles. 

Instruction 

7. The instructional program is derived from and linked to specified com- 
petencies. 

8. Instruction which supports competency development is organized into 
units of manageable size. 

9. Instruction «s organized and constituted so as to accommodate learner 
style, sequence p'^eference. pacing, and perceived needs. 

10. Learner progress is determined by demonstrated competency. 

11. The extent of learner's progress in demonstrating competencies is 
made known to him throughout the program. 

12. Instructional specifications are reviewed and revised based on learner 
feedbeck data. 

Assessment 

1 3. Competency measures are validly related to competency statements. 

14. Competency measures are specific, realistic, and sensitive to nuance. 
14.1 Procedures for measuring competency demonstration assure 
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quality and consistency. 

14.3 Competency measures allow for the influence of setting varia- 
btes upon performance. 

15. Competency measures discriminate on the basis of standards set for 
competency demonstration. 

16. Data provided by competency measures are manageable and useful 
in decision making. 

17. Assessment procedures and criteria are described and made public 
prior to instruction. 

6ov0rnance and Management 

16. Statements of policy exist that dictate in broad outline the intended 
structure, content, operation and resource base of the program, in- 
cluding the teaching competencies to be demonstrated for exit from 
the program. 

19. Formally recognized procedures and mechanisms exist for arriving 
at policy c'ecisions. 

19.1 A formally recognized policy making (governing) body exists 
for the program. 

19.2 All institutions, agencies, organizations, and groups partici- 
pating in the program are represented in policy decisions that 
affect the program. 

19.3 Policy decisions are supportea by, and made after consideration 
of, data on program effectiveness and resources required. 

20. Management functions, responsibilities, procedures, and mechanisms 
are clearly defined and made explicit. 

20.1 Management decisions reflect stated program philosophy and 
policy. 

20.2 The identified professional with responsibility for decision has 
authority and resources to implement the decision. 

20.3 Program operations are designed to model the characteristics 
desired of schools and classrooms in which program graduates 
will teach. 

20.3a Job definitions, staff selections, and responsibility as- 
signments are linked to the management functions that 
exist. 

20.4 Formally recognized procedures and mechanisms exist for 
arriving at the various levels of program management decisions. 

Staff Development 

21. Program staff attempt to model the attitudes and behaviors desired of 
students in the program. 

22. Provisions are made for staff orientation, assessment, and improve- 
ment. 
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23. Staff development programs are based upon and engaged in after 
consideration of data on staff performance. 

Total Program 

24. Research and dissemination activities are an integral part of the total 
instructional system. 

24.1 A research strategy for the validation and revision of program 
components exists and is operational. 

24.2 A data-based management system is operational. 

24. 3 Procedures for systematic use of available data exist. 

25 institutio ^al flexibility is sufficient for all aspects of the program. 

25.1 Reward structure in the institution support CBTE roles and 
requirements. 

25.2 Financial structure (monies and other resources) in the system 
support collaborative arrangements necessary for the program. 

25.3 Course, grading, and program revision procedures support the 
tentativeness necessary to implement the program. 

26. The program is planned and operated as a totally unified, integrated 
system. 



FOOTNOTES 

' Stanley Elam. P«fformanco*Basod Toach»r Education. What is the State ot the Art?" 
(Washington, D C AACTE. 1971) 

' Allan Schmiedar. Competancy-Basad Education The Stata of tha Scana." (Washinoton. 
0 0 AACTE. 1973). pp 10-11. Susan S Sharwm. Paftormanca-Basad Taachar Education. 
Results of a Recant Survey ' (t'rinceton. N J Educational Testing Service. 1973); Donald W. 
McCurdy. Status Study ot Competency Based Teacher Education Programs in Science. " 
(Paper presented at the Assoctation tor the Education of Teachers m Science. March 15. 1974). 

» Syracuse University— James Collins; Oregon— H. Del Schalock; Michigan State University 
—J Bfuce Burke: University ot Georgia— Qilbert Shearron and Charles E. Johnson; Florida 
State University —Norman Dodi; Columbia University Teachers College— Bruce Joyce; Uni- 
versity of Wisconsin— M Vere DeVault; University of Toledo --Qeorge E Dickson. University 
of Houston -James Cooper. Wilford Weber, and W Robert Houston James Steffensen and 
Allen Schmieder represent USOE and John Hansen is Executive Secretary. , 
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Evaluating Ways to 

Help Teachers to 
Behave Desirably 



N. L Gag« 



m Its Simplest form, the logic of teacher education depends on two kinds 
of relationships, in the first kind, the behaviors and characteristics of 
teachers are the independent variables, and measures of student achieve- 
ment of educational objectives are the dependent variables. After causal 
relationships of this kind have been established, we know what kinds of 
teacher behavior and characteristics are desirable, m the sense that they 
facilitate student achievement. Then we are logically ready to go on to the 
second kind of reiationship-^that between teacher education programs and 
procedures. sefv:ng as the independent variables, and teacher behaviors 
and characteristics, serving as the dependent variables. 

Logically, the first kind of relationship should be discovered and estab* 
hshed before we can undertake to establish the second kind of relationship, 
Simply because we do not know what kinds of teacher behaviors and char- 
acteristics we should aim at m our teacher education programs until the 
first kind of relationship is known But it is obvious that we cannot afford to 
be completely logical m this sense- Children must be taught; hence, teach- 
ers must be educated And neither enterprise can wait, nor has it waited 
Over the centuries, for scientifically established evidence to appear on the 
kinds of teacher behaviors and characteristics that best facilitate student 
achievement 



Before dealing with the second kind of relationship—that between teach- 



The GeneralizabilHy Issue 
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er education procedures and teacher behaviors or competencies— let us 
examine a problem that has vexed thoughtful students of the first kind of 
relationship. This is the problem of whether it will ever be possible to deter- 
mine Such relationships with enough generality for any practical value in 
teacher education. The issue has been around for a long time. In 1961. 
Bantock wrote that 

... it is only possible to construct a model of successful 
teaching of a particular subject in terms of insight into the 
nature of that particular subject" and of the particular sorts 
of demands that the nature of the "subject" makes in rela- 
tion to the stage of development of the pupils concerned.^ 

Thus Bantock was arguing thai .inot hope to find a general rela- 
tionship between a teacher behavior variable and student achievement, but 
only a relationship that is Specific to a given subject matter and a given 
stage of student development. Since there are many subject matters and 
many stages of student development, there will be many possible relation- 
ships between any particular kind of teacher behavior and student achieve- 
ment. If we consider only five subject matters (for example, mathematics, 
science. English, social studies, and foreign languages) and four stages of 
student development (i e . primary, elementary, junior high school, and 
senior high school), we have 20 possible combinations of subject matter 
and student development and thus 20 possible kinds of relationships be- 
tween any particular kind of teacher behavior and any particular kind of 
student achievement. Consider, for example, the teacher behavior variable 
called frequency of "disapproval" and student achievement as measured 
by a test of knowledge and understanding, adjusted for preinstructional 
student aptitude. We cannot hop& for any general relationship between 
these two variables if Bantock is correct. The relationship may be positive 
for primary grade arithmetic, negative for senior high school English, and 
zero for elementary school foreign languages. 

This kind of argument has been made more recently by Heath and Niel- 
sen (1973) in their critique entitled 'The Myth of Performance Based Teach- 
er Education. "2 These authors found much fault with the volume entitled 
Teaching Behaviors and Student Achievement in which Rosenshine (1971) 
reviewed more than £0 correlational studies. ^ It was made even more re- 
cently, and in especially graphic form, by Maxwell (1974). from whom I 
cannot resist quoting at some length: 

Despite the fact that proponents of PBTE readily admit 
that no one has ever identified the competencies of the 
successful teacher, little thought seems to have been given 
to the question of why this is indeed the case. There seems to 
have been remarkably little understanding of the natural 
science setting from which the basic model was borrowed. 

The natural sciences can be rigorous because they deal 
with simple phenomena. They can deal in simple system 
models t>ecause they can safely assume homogeneity. One 
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electron can usually be assumed to be like every other 
electron 

But suppose that in a simple circuit each electron differed 
from every other, so that some wished to move from t^egative 
to positive, some from positive to negative, and some not at 
all. when the battery was hooked up. Further, suppose that 
some were pleased when a firm hand hooked up the battery 
while others preferred a more gentle touch — and that those 
who preferred the firm hand did so most strongly when 
firmness seemed somehow to suit the person displaying it, 
in the eyes of the particular electron making the judgment— 
and that the same individuality of judgment characterized the 
electrons who preferred the gentle touch. 

If this were indeed the case, what would we say about the 
outcome of hooking up a bulb, some wire, and a battery? 
Obviously, we could make no simple generalization about so 
complex a situation. 

Vet the analogy far understates the complexity of the stu- 
dent/teacher relationship. The truly remarkable thing is that 
those one presumes to be students of this phenomenon would 
recognize so little of its complexity. We don*t have a list of 
the competencies of the successful teache** for the simple 
reason that teaching is an art— not a science A behavior 
that IS a valuable asset to a given teacher with a certain 
personality m many (but not all) situations, insofar as most 
(but not all) of his pupils are concerned, may well go over 
like a lead balloon (to most students) if affected by some 
other teacher with a different personality/ 

Bantock. Heath and Nielson. and especially Maxwell are raising no trivial 
question It they are right, the search for general, or even moderately gen« 
eral. relationships between teacher behavors and student achievement 15 
indeed futile But they have raised this question on a prion grounds, as if 
the answer were foreordained by the sheer logic of the problem And what 
they offer as a logical possibility is not necessarily an empirical fact. It was 
just this point that was made by Martin (1963) m a paper entitled Can 
There Be Univei sally Applicable Criteria of Good Teaching*?'"^ Attacking 
the issue as an analytic philosopher, she pointed out that the fact of dif- 
ferences between two subject matters does not preclude the possibility of 
similarities m the best ways of teaching them Whether generalizations 
about teaching methods across subject matters are possible is an empin* 
cal. not a logical, question or one to be settled by conceptual analysts. She 
concluded that Bantock s reasoning (and. by the same token, that of Heath 
and Nielson and of Maxwell) should not make us halt inquiry aimed at 
determining general principles of teaching. 

Present Knowledge on What Teaching Behavior Is Destratrie 

05 cours^e. that .nquiry has not been stopped by these arguments. Despite 
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all the methodological flaws in the research reviewed and m the reviewers' 
methods of reviewing that research. Rosenshine (1971) and, more recently. 
Ounkin and Biddte (1974)^ have found some relationships that hold up 
across a variety of subject matters, grade levels, and other contexts. For 
example, in Rosenshine*s review, of 17 studies dealing with the disapproval- 
achievement relationship. 12 yielded negative relationships and only 2 
yielded positive relationships. The other three studies yielded mixed or 
doubtful results. The antigenerality argument would hold that such results 
should be checked to see whether they still occur when certain controls are 

applied. In a recent note entitled "Did You Control for ?" Rosenshine 

reported what he found when he organised his review of various teacher 
behaviors according to type of outcome measure, grade level, subject 
matter, and specification of the observational variable, i.e., the operational 
definition Of the teacher behavior. He reported that The results were nof 
any different from those already presented. That is. the overall results on 
teacher questions, for example, are unchanged if one divides the tables 
on questions according to outcome measures; the results on clarity or on 
task-orientation appear significant across all grade levels and all subject 
areas reported." 

I am not saying that many generalizations are weiUestablished. Rela- 
tionships between teacher behavior and student achievement may still turn 
out. as Rosenshine is aware, to be a function of grade level, subject matter, 
type of achievemen:. urban vs. rural location, sex of student, sex of teacher, 
socioeconomic status of the student, ethnic variables, the 3tudent*s **iearn*^ 
ing style/' and so on But the possibility of such phenomena, which the 
statistician would call fnteractions. does not establish their actuality. Only 
research can settle the issue. 

In other fields of the behavioral sciences, certain relationships have held 
up over enormous ranges of contextual variation. For example, {he relation- 
ship between scholastic aptitude and scholastic achievement has always 
been found to be positive m thousands of studies ranging in grade level from 
preschool to graduate school, from children on welfare to children m private 
schools, in every academic subject, in every region, and in every ethnic 
and sociological category. Similarly, sex differences in dominance and 
aggressiveness have always been found to fall in the same direction regard- 
less of age. social class, and a host of other variables.' Certain behavior 
modification techniques, such as those used in treating bedwetting or re- 
ducing Classroom misbehavior, have been found to be effective in widely 
differing age levels and social classes. For example, removing attention 
from misbehaving children has decreased tho'r misbehavior regularly. (See 
various issues of the Journal ot Applied Behavior Analysis. ) 

The possibility of generalizations across subject matters deserves a bit 
of special attention, for it seems especially questionable. Can a good method 
of teaching a concept in geography also be a good method of teaching a 
concept in art*? Can teachei^ behaviors conducive to understanding a prin- 
ciple in economics also facilitate understanding a principle in physics^ 
For the psychologist of learning, the answer is. of course, affirmative. For 
him. a concept is a concept and a principle is a principle in very much a 
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umvetsal fof m i ei^.Uilioss o\ .u i idents of subiect matter AnO concepts .md 
prmcipies are learned, and hence c.if^ be^st be tiiuQht. m much the same 
way regardless ot iiubiect Duitter Thus learning the concept of peninsula 
in qeocraphv »s similar to learnmq the concept of asymmetry m art. and 
teachincj the economic principle ot supply and demand is much the same as 
teaching the pnrKiple o! physics caned Ohms Law M least that »s the 
assumption guiding the search for gerterai principles ot effective teaching 
or teacf^mg competencies That search has not yet gone on long enough, 
nor has it tailed so dismaiiv as to lustify as be»ng caiied off Senous research 
on teaching has only a relatively short history, as aiso does serious educa- 
tional research of any kind it was seldom performed on any adequate scale 
before about 20 years ago its increase m quantity is Still fa» from adequate 
if we can )udge by the number of studies that Rosenshme and Dunk»n and 
Biddie were able to fmd for their reviews 

Furthermore, most of the studies of .eacher competency have dealt wtth 
only univariate relationships That is. only one kind of teacher behavior at a 
time has been correlated with a measure of student achievement it is httle 
wonder that the r^^ignitude of the average correlation coefficient for any 
Single kind of teacher behavior has been small Indeed, we should be sus- 
picious if the coefficients had been high because, by everything that seem.s 
plausible, a smgie kmd of teacher behavior variable should make only a 
small difference m student achievement We need studies m which patterns 
of two. three and more kinds of teacher behavior are studied Those pat- 
terns should not be studied only by means of multivariate statistics or 
multiple correlation coefficients They should also be studied as sequences 
in the fofiris recently Sketched by Flanders The kmds of transition chains 
he described as anaiy/abie by computer programs now available should 
make It possible to analyze teaching with something approaching adequate 
sophistication Our counts of the frequencies of smgle kinds of behavior, 
such as disapproval are analogous to doscnbmg a piece of music by 
counting the number of F-sharps it contains We know that the melody re- 
sides not in the frequency of single >o?os. but m the relationships between 
r)otes The Significant dimensions of teaching may sim.larly be found to 
reside »n the patterns. s<»quences. or chains of teacher and Student behavior 
that occur m the classroom if so. these new approaches to describing 
classroom pher imena may begm to yield more substantial explanations of 
the variance m achievement between classrooms when student ability is 
held constant 

So far. I have defended the search for the kmd of relationship that wiH 
make performance based teacher education truly advantageous i have 
defended the search against logical or conceptual attacks, against method- 
ological criticisms, against premaNre discouragement, and agamsl com- 
placency about current approaches. 1 believe v^vformance based teacher 
education should go on. even before tne research successes that we seek 
have been won. because even without the fruits of such research it provides 
a tremendous heuristic advance over conventional teacher education It 
exposes the quest ons we need to ask. and it proceeds m an optimal way 
on the basis of w^at knowledge about teacher competencies is now avail- 
able 
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Research on Teacher Education 

But now let me tui»i to the second kind ot relationship- - that uetween 
teacher education procedures and methods, on the one hand, and teacher 
competencies, on the other frnow can we know when a teacher education 
method »s producing a teacher competency, or a desirable kind ot teacher 
behavior, or the use ot good models or methods ot teact-ing. or what Gagn^ 
(1973) recently identitied as more importa*'t. m his opinion, than any ot 
these, namely events that support learmn^j processes ' in short, how 
can we evaluate the mstri^-Nonal strategies that are used *ii teacher edu- 
cation programs'' 



**Knowlng That** and "Knowing How** 

My answer is based m part on i? distinction between two K<nds ot com- 
petencies This IS tne distinction tust made, as tar as I know, by Ryle ( 1949) 
and since then further analyzed by Roland M961)." among others— the 
distinction between knowing that" and knowing how " The tirst Kind ot 
Knowledge, knowing that." reters to the ability to state tactual propositions, 
such as knowing that Skinner coined the term "operant ' or that teacher 
education is a controversial subject Knowing how." on the other hand, 
refers to sKilis or o,>'3rat.'on$. and Roland added the further qualification that 
the distmguithing feature of such sKills is that they imply learning through 
practice As she put a. Jones could not Know how to swim or speaK 
French unless he had at some time practiced swimming or tried to speaK 
French 

My purpose m referring to these distinctions is to delimit the Kinds ot 
outcomes or objectives ot teacher education with which i shall deal. In 
some conceptions of competency, or performance, b^sed i^richer educa- 
tion, all outcomes are embraced. Such conceptions 'e^^ • • s 
general or liberal education, the teacher's Knowledge o* i -.v ..u / • »t ill 
teach, the teacher's Knowledge about curriculum and mstr jc*»o»^ Mm f ach- 
er \ Knowledge about the social, historical, and phil m 'r. j.iaat'*' ns 
ot education, and finally, the teacher's knowledge ot hns^ * 

1 place most of these Kinds ot Knowledge in Ryie s k th.U" 
category i also prefer to omit from my discussion any consider <tion ui ♦/ays 
ot helping teachers to acquire such Knowledge and ways ot evaluating tech- 
niques of giving teachers such Knowledge. The reasc tor restricting dis- 
cussion in this way is twofold First, ways of inculcating i-nd evaluating such 
knowledge are relatively well established Teacher educa: on s mam weaK- 
nesses. m my opinion, have not been located in this realm. W.^cr* it comos 
to giving teachers Knowledge of this Kind, we have done about as well or as 
poorly as the rest ot higher education. Second, many ot these Kinds of 
Knowledge are provided by parts ot the university outside of schools or 
colleges of education, and p'^jfessional educators can impose performance 
based approaches outside their own domains only at the risK of arousing 
consider .lOie academic opposition My inclination is to try to establish 
the value of perform a nee. based teacher educatjn in those domains tor 
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which professional educators ar^ primarily responsible. And, of these, the 
one that provides the greatest room for improvement and the greatest 
opportunity is the realm of "knowledge how"— the realm of skills and habits 
•n teaching. This is the realm in which we include what the teacher does In 
making decisions about how to teach and how he then uses the skills nec- 
essary to carry out those decisions. "Knowing that ' is illustrated by the 
student s ability to state and even explain Archimedes* and Newton*s laws. 
"Knowing how" is illustrated by the student's swimming. Although swimming 
is explained by Archimedes' and Newton's laws, knowing those laws is one 
thing, while knowing how to swim is another. The two kinds of knowledge 
are independent of one another. Similarly, knowing the laws of learning and 
being able to expound operant conditioning or cognitive theory is one thing, 
but even though those tneoriejs deal with what makes certain kinds of teach- 
ing possible, knowledge of those theories is not sufficient to enable one to 
teach. 

It is this realization mat has led teacher educators to include student 
teaching, or some other kind of practice, in their programs. Teacher educa- 
tors may not have beer aware of Ryle's distinction between knowing that 
and knowing how, or the many subtle eiaboraiions of thai distinction that 
have been formulated, but they have acted upon that distinction. It may be 
that distinction which most clearly lays the basis for our whole concern 
that teachar education be performance based. We want teachers to know 
how to teach, not meiely to know that certain things aboui teaching are 
true. Thi* realization has also led to all the attempts of the last decade to 
improve the effects of practice for prospective teachers-— attempts in the 
form of microteaching. minicourses. and the hundreds of teacher training 
products that have been developed in recent years. 



How to Evatuatt Teacher Training 

How should we evaluate the strategies that we develop to improve the 
degree to which teachers behave desirably? In dealing with this topic, we 
must set up hypotheses about desirable ways of teaching. Furthermore, we 
must set up hypotheses about strategies for getting teachers to behave in 
these desirable ways. Both of these kinds of hypotheses have frequently 
been made in the developmental work of the last few years. 



Teacher Training Producte 

That inference seems justified by the plethora of teacher training products 
that have been developed, identified, and catalogued by organizations in 
Florida, at the University of Houston, and at Stanford University. In the 
Stanford catalog, which had the benefit of being based upon several prede- 
cessors developed elsewhere, more than 600 such products have been 
described in terms of more than 100 descriptors, each or which has provided 
the basis for an inde^' of the catalog. The Stanford catalog is stored on 
computer tape, and information about teacher training products with any 
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given characteristic or set of characteristics can be readily obtained. Thus 
products intended to improve (a) question asking skills in (b) secondary 
school (c) mathematics teaching, can be identified quickly. 

One descriptor in the Stanford Catalog of Teacher Training Products was 
"References describing field test results.*' Of the 657 products described 
in the catalog, only 87, or 13 percent, came with references to published 
field test results. For another 83. or 13 percent, of the products, the source 
catalog stated only that the product had been "field tested by the developer.** 
For 22. or 3 percent of the products, reference was made to unpublished 
data or an unpublished report. When the developers were asked for infor- 
mation by mail, positive results were claimed for 7. or about 1 percent, of 
the products. For 12 of the products, the developer indicated that field 
testing was planned or in progress. For 77 products, no field testing had 
been performed, or it was indicated that the results were available only to 
the developer of the product. Finally, for 369. or 56 percent, of the products, 
no information concerning field testing was available in any form. 

Thus fd^ we have not determined what kind of information was provided 
in the 87 references to published field tests. Nor have we ascertained the 
bases for the developers* claims of positive results for seven of the prod- 
ucts. Nonetheless, even without further investigation and analysis, the broad 
outlines of the field test information indicate that data are available in too 
few cases, m too little detail, and in a form too unavailable for evaluation 
by potential users or critics of the product. 

The field testing of teacher training products can be expensive and diffi* 
cult. In many cases, it r.i<«y not seem important enough to justify the cost 
in view of the nature of the product and past experience with products of 
Its kind, such as manuals or programmed instructional materials. The field 
testing of a product for validation can cost much more than the development 
of the proJuct itself. 

Yet it must be clear that the present state of affairs should not be allowed 
to continue much longer. Standards should be developed in this field com- 
parable to the ethical and professional standards that have been promul- 
gated by the American Psychological Association (1966) for psychological 
tests and manuals. 



EvaJuative Experimentation 

What form should such standards4ake? How should strategies for incul- 
cating teaching competencies be evaluated? To a considerable degree, the 
broad outlines of answers to these questions are already available. In 
general, we can evaluate such products or strategies by using them as 
independent variables, or treatments, in experiments or quasi-experiments. 
Teachers or prospective teachers given training with the procedure or prod* 
uct should be compared with teachers or prospective teachers not given 
such training. Designs for such experiments were clearly set forth by 
Campbell and Stanley (1963)^^ and have recently been treated with specific 
reference to the problem of evaluating teacher training products by Okey 
and Ciesia (1973).'^ The latter exposition focused particularly on three 
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quasi-experimental designs because of the presumed difficulty, in research 
with teachers, of obtaining control groups of teachers. These three quasi- 
experimental designs are the time-series design, the equivalent time- 
samples design, and the posttest-only control group design. 

These matters of design are well known. And the latter three designs 
have been well illustrated m their application by Okey and Ciesia to the 
evaluation of an actual teacher training product, namely, their own Teaching 
for Mastery self-instructional program. So I shall not deal further with these 
aspects of the problem of evaluating ways to get teachers to behave desir- 
ably. Let me turn instead to three aspects of the evaluation problem that 
are. as I see it. somewhat less well known and understood. These are (a) the 
use of criteria at various levels on the continuum of immediacy-ultimacy, 
(b) the use of unobtrusive measures, and (c) the use of factorial designs. 

My first point is aimed against a dogma which appeared as a reaction 
against the nonuse of student achievement as a criterion in studies of teach- 
ing. Until the 1950*s. criteria of teaching effectiveness usually took the form 
of ratings by supervisors, principals, experts, or students. The bibliography 
by Domas and Tiedeman annotated more than 1,000 studies, of which only 
a few used student achievement as the dependent variable.^' In 1954. 
Ackerman published a review of the few available studies that related 
teacher competence to pupil change. and he was followed in 1956 by 
Mitzel and Gross with a similar review.'^ Since then, perhaps 100 studies, 
mostly correlational studies, of relationships of teacher and teaching vari- 
ables to student achievement have appeared and have been reviewed in 
the volumes by Rosenshine and by Dunkin and Biddle (1974). More studies 
of this kind are needed— many more. 

But my point is that these are not the only kinds of studies worth doing. 
Not everything desirable about teaching or teacher education needs to be 
justified on the basis of its relationship with student achievement. Achieve- 
ment is at one end of a causal chain, or at one corner of a causal network. 
Many desirable factors in teaching or teacher education may be too remote 
from that final hnk in the chain to have a discernible effect upon it. They are 
desirable, nonetheless, if they can be shown to have good effects on links 
closer to the treatment, or independent variable, that we are evaluating. 
In medicine, not every feature of the physician*s technique is validated on 
the basis of its correlation with patient recovery or longevity. It would be 
difficult to find such a correlation for every detail in methods of taking blood 
pressure, using an opthalmoscope. swabbing a throat, applying a plaster 
cast around a broken leg. sewing a suture, or massaging a muscle. The 
criteria for the effectiveness of such competencies on the part of physicians 
are not patient longevity, or even patient recovery, but rather outcomes 
much closer in the causal chain to the particular competency in medical 
practice. Does the throat swab cover the tissue it should cover? Does the 
plaster cast endure long enough without being too heavy? Does the suture 
hold the wound together and leave a minimal scar? Does the massage re* 
store Circulation and relieve an ache? 

By the same logic, we should ask of a technique of teacher education not 
only, and not always, whether it results in improved student achievement, 
retention, and even transfer of cognitive and social-emotional objectives. 
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We Should asK first, perhaps, whether the teachers being educated with the 
technique have some respect and some liking for it. Unless they take some 
pleasure in the technique, intellectual or social pleasure, or both, the 
teacher training method will probably not be effective, because it will not 
be used. Second, we should ask whether the teacher training method has 
the Intended effects on teacher behavior or performance, either in the inter* 
active situation with students or in the teacher's preparation, planning, 
and decision making before he meets his students. If teacher attitude and 
teacher behavior are influenced as intended, we are justified in regarding 
the teacher training as validated in an important sense. We should not 
insist that every single procedure or product for training teachers be 
validated by its effects on student achievement. 

Further, just as criteria come in hierarchies and not only those at the top 
are worth using, so settings for training teachers come in a hierarchy of 
realism. We can go from training animals to tutoring individual students in 
the laboratory to teaching groups of 5 to 10 students in a microteaching 
setting, and perhaps other intermediate settings, before we come to real 
classes in real schools. Not every product or procedure should be required 
to prove its worth on the basis of its effects in actual classrooms. Here 
again we should try to make reasonable inferences in a graduated sequence. 

In short, proximate criteria of all sorts are justified. We are likely to make 
many errors in which we reject good methods as ineffective, if we always 
demand that every teacher education procedure or teacher training product 
result directly in improved student achievement. 

Let me turn now to the use of unobtrusive measures. The term was origi- 
nated by Webb and others in a 1966 volume by that title. It refers to **social 
science research data not obtained by interview or questionnaire.**^^ Re- 
search on teaching has not been as prone as other kinds of behavioral 
science research to depend entirely on questionnaires and interviews. The 
observation of social interaction, especially in the classroom, has been 
more highly developed in research on teaching than in other fields of the 
behavioral sciences. Nonetheless there is need for additional ways of 
measuring the outcomes and processes of teaching and of teacher training, 
especially in the non*cognitive domains. We need ways that are not subject 
to the biases and weaknesses of tests, questionnaires, and interviews. 
Webb and his co-authors gave a bookful of examples of unobtrusive mea- 
sures, which they classified as those depending on (a) physical traces of 
erosion and accretion; (b) the use of archives, including running records 
of various kinds, as well as episodic and private records; and (c) observe- 
tion. either simple or contrived. 

Of these, research on the evaluation of teacher training strategies has 
already used much simple observation and some contrived observation, 
depending on hidden hardware and controls, such as remote-controlled 
videotape recorders. But it has made too little use of the other kinds of 
unobtrusive measures. I can offer only a few examples of what 1 have in 
mind, if a helicopter flies over the playground of a multiracial school and 
photographs the upturned faces, the picture will yield an unobtrusive 
measure of racial integration in the playground. (1 am indebted to Harold 
B. Gerard for this example.) Similarly, if the use of a minicourse is made 
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completely voluntary, how many teacher trainees even look into it. and 
how many voluntarily complete it? If a teacher education program is 
changed, what happens to the characteristics of those who apply for admis- 
sion to it--&uch characteristics as prior grade-point average, sex ratio, 
ethnic and socioeconomic background, and scholastic aptitude? What 
happens to the longevity in the profession of its graduates? What happens 
to the incidence of broken windows and other kinds of vandalism if teachers 
are trained in methods of enhancing the students* sense that they control 
their own fate? What can we tell about a reading teacher's effectiveness 
from an unobtrusive count of the number of books checked out of the library 
by his students on their own initiative? What can we tell about the interest 
of a teacher's choice of topics, before and after the teacher has been given 
a relevant minicourse. by unobtrusively listening to students' conversations 
as they leave the class or ride home on the school bus? What can we tell 
about the effectiveness of a teacher's unit on nutrition by watch*ng what 
his students buy in the school's cafeteria, store, and vending machines? 
My examples may be inadequate, but the point remains. We could benefit 
from greater ingenuity in evaluating teacher training strategies through 
supplementing present methods with ethically but effectively usea unobtru* 
sive measures. 

Finally. I turn to the question of how to evaluate a teacher training system 
as a whole while also evaluating its parts. This kind of problem arises when 
we want to know not only whether the System or program as a whole has 
made a difference, but whether each of its components carries its weight 
«nd even whether the effectiveness of the whole system is greater, through 
some kind of interaction among the parts, than the sum of the effects of the 
parts. The classic solution to this problem is factorial design. For example, 
if there are two minicourses. A and B. there can be four treatment groups, 
namely teachers who take A only, those who take B only, those who take 
both A and B. and those who take neither. But if there are three mini- 
courses, or other components, in a training system, the number of possible 
treatment groups becomes 8; with 4 components. 16. and so on. From such 
a design, one can learn about the so-called main effects, that is. the effects 
of each component by itself, and also about interaction effects, whereby 
two or more components together have an effect different from the mere 
sum of their separate effects. 

For complex teacher training systems, complete factorial designs are un- 
workable simply because they require too many teachers. Further, they 
would yield much unpromising information, such as information about 
interaction effects that are highly unlikely to occur or too hard to interpret 
if they do occur. For these reasons, statisticians have invented confounded 
factorial designs, fractional factorial designs, and incomplete block designs. 
The details of these designs are beyond the scope of this paper. But research 
on teacher education should eventually make use of these designs, which 
sacrifice some of the combinations in systematic ways and some of the 
results on interactions, while becoming much more manageable. 

i know of only one example of this kind of research. It dealt with an 
elemental analysis of the Keller method of instruction. Calhoun determined 
whether six elements of the Keller plan made a difference in student 
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achfevemeni in a i.ompiete factorial destgn. he would have had 2*^. or 64. 
combinations ot tteatmonth ifisteail. he had only six groups, and he s«icri- 
f»ced the opportunity to team about many possible interactions But he was 
able to coticiude that each of the four principle elements of the Keller plan 
contributed to the overall effectiveness of the method. These elements 
were ui) using proctors to test students, ^b^ using a unit-perfection criterion 
over small units of material, ic) giving immediate feedback as to perfor- 
mance, and (d) allowing students to go at their own rate Further, he found 
that the addition ot lectures did not improve achievement. ' 

The same logic and general design can be applied to the evaluation of 
systematic teacher training models of the kind being developed at the 
Stanford Center for Research and Development m Teaching. We hope to be 
able to evaluate m this way not only the effectiveness of the system as a 
whole but that of each of its components. I commend the approach to any- 
one concerned with the evaluation of instructional strategies m performance 
based teacher education 



f OOTNOTbb 

G H Bantock Educ.ii'on.ji Research A Cft-cism Harvard Educational Revtew. XXX i. 
I Summer 196*1 264.^60 

R o He:«h .iMO M A N.eison Tf)e Myth of Performance Based Teacher Education 
iBefKeiey C.'*i Nomos mstiiuii- i973> 

BufaK Rosens^.ne Teaching Behavours dnd Student Achievement i London. Eng Nat«o.idt 
Foundation for Educai'onai Research m England and Wales. 197 1 1 

' W 0 Ma«wen PBTE A Case oi the Emperor s New Clothes Pht Delta Kappan. LV. 5 
iJdnuarv 19/4i 306 3t1 

J R Wurt«n Can Tnere Be UniversaUy Appncaoie Criteria o* Good Teaching'' Harvard 
Bducat'onat Review XXX;.. {Fan I963i.4a4 49i 

• Mu-.naei Dunk in and Bruce Biddie The Study of Teaching iNew York Holt. Rmehart and 
VV.nston I9r4, 

E Maccobv ed . The Study of Se* Differences iStaniord Cai Stanford University Press. 
19661 

rsi A FJonde»s ^ne Chanqmq Bases or Periorrriance Based Teaching. Phi Delta Kappan, 

LV 5 January Hr4i 31? 316 

R M Gagne Obsefvations or School Learning Educational Psychologist. X <l973i. 
lib 

G'.»jertRvie Th^ Concept of Mmd iKevn y or k Barnob 4 Nobie. 1949J 

j.inf* RoMnd On the ReOuCI'On o* Know.ng That to Knowing How. m Language and 
Coniepts in fduca/'on. B O Smith and R h Enn.s. eds iChioago Rand McNaiiy. 1961). 
pp S9 ri 

loid p 63 

• En.s Sando/ CBTE The Nay^ o* Texas. Phi Delta Kappan. LV. f, (January. 197^ 304- 

306 

Vaxweii PBTE a rase o* the Emperor s Ne* Clothes 

• Standards tor Educational and Psychological Tests and Manuals (Washington. D C 
American "Psychological Association. 1966) 

• O T Campt)e<i and J C Stan'tv Experimental and Ouasi-Exportmontai Designs tor 
Re* ;rch on Teac^-ng. <n Ha .ihook ot Research on Teaching. N l Gag© ©d (ChicaQO 
Band McNaiiv. 19631 pp 171-246 



ERLC 



184 



BEST COPY AVAILABLE 



J R OHey and J t Ciesw, Designs to? the Evaluation o* Teacher Training Materials. ' 
AV Comtnunicatton R9¥mw. KXi. (Fall. I9;3». ?99.3i0 

" S J Domas and 0 V Tiedoman. Teacher Competence An Annotated Bibliography /' 
Jcum^l of ExpertwentMl Education. XiX. (December. i960k I0i-218 

•« W I AcKerman. Teacher Competence and PupH Change. * Harvard Educational Roview, 
XXtV. (Fall. 1954). 273-289 

'♦HE Mit^el and Cec»iy F Gross. A Oiticaf Review of the Development of Pup^i Growth 
Cnt9rfa m StudfS of Teachet Btt9Ctivene&& (New York Board ot Higher Education of the City 
ot New York. Dwision ot Teacher Education. Ottice of f^esearch and Evaluation. 1966) <Re* 
tear Oh Series 31 . mimeographed ) 

E J Webb. 0 T Campbeu. R D Schwart2. and Lee Sechrest. Unobtru&tve Jtreasi/res 
Nonreactive Research m the Social Sciences (Chicago Rand McNaiiy. 1966). p i 

J r Calhoun. Elemental Analysis of the Keller Method of instruction Paper presented 
at American Psychological Association. Montreal. 1973 



ERIC 



185 



BEST COPY MIUBLE 




Task Force on 
Evaluating 
Instructional 
Strategies 



WKf Of d A. W0b«r 
H%6 Fl«nd«rt 

This paper briefly describes the efforts of the Conference task force which 
was concerned with issues relating to the evaluation of instructional strate* 
gtes. Specifically, the task force: (t) sought to identify the most pressing 
of those problems associated with the evaluation of instructional strategies; 
and (2) offered very tentative suggestions regarding solutions to the prob- 
lems it identified. Because of the diverse nature of the task force's member- 
ship, its efforts reflect three separate but related $3ts of viewpoints and 
concerns with regard to the evaluation of instructional strategies: (1) the 
viewpoint of the educational researcher who is primarily concerned with 
establishing empirically validated relationships between specific instruc- 
tional strategies and particular learner outcomes: (2) the viewpoint of the 
teacher educator who is primarily concerned with: (a) identifying instruc- 
tional strategies which have been proved effective; (b) helping teacher 
education students and teachers to acquire and demonstrate those instruc- 
tional strategies; and (c) utilizing assessment procedures which determine 
the extent to which a student has successfully demonstrated the ability to 
effectively use those instructional strategies; and (3) the viewpoint of the 
public school person — teacher or administrator — who is primarily concerned 
with identifying and using instructional strategies which have the demon- 
strated potential to effectively and efficiently facilitate pupil learning. 

As might be expected, given the state of the art and the time it was al- 
loted. the task force found the identification of problems related to the 
evaluation of instructional strategies to be far easier than the positing of 
solutions. Generally, nearly all of the solutions which were suggested con- 
sisted of recommendations that a particular problem be given careful 
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attention. That is. it was suggested that educational researchers should 
direct their efforts toward finding solutions to each of a number of problems 
identified as crucial to the evaluation of instructional strategies. The dis- 
cussion which follows briefly describes the Six major problems identified 
by the task force and notes the solution which was offered. 

Operational Definition, The task force noted that the term instructional 
strategy is rather ambiguous. That is. the term lacks a commonly used, 
empirical, conceptual or operational definition. It was claimed that edu- 
cational researchers have been far from clear as to the phenomena which 
constitute an instructional strategy. On the one hand, there are those who 
define an instructional strategy as a rather simple teaching skill, such as the 
asking of probing questions. On the other hand, there are those who view an 
instructional Strategy as a very complex set of instructional behaviors, such 
as individualizing instruction. This lack of clear understanding as to what 
behavioral phenomena constitute an instructional strategy creates a great 
deal of difficulty with regard to research efforts which focus on instructional 
strategies. Consequently, researchers seem to be rather unsure as to the 
unit of measurement to be employed; that is. should their research narrow 
in on very discrete, simple teaching behaviors or should they be concerned 
with more complex sequences of teaching behavior? It was suggested that 
presently, there appears to be a lack of empirical evidence to suggest the 
unit or units of measurement that are m^st appropriately used in studying 
instructional strategies. It was recognized, however, that recent attempts 
to identify those teaching behavior sequences and instructional strategies 
which Constitute various models of teaching hold promise because these 
conceptualizations should foster operational definitions which will permit 
better decisions concerning questions of unit of measure. Consequently, it 
was the suggestion of the task force that educational researchers who are 
concerned with the evaluation of instructional strategies begin by focusing 
their attention on the identification of models of teaching which are pre* 
dieted on assumed relationships between particular sets of teaching be- 
haviors and the learning outcomes that they are expected to facilitate. 

Goodness Versus Ettectiveness, The task force expressed concern 
that the study of instructional strategies should focus on a determination of 
those instructional strategies which are most effective rather on those 
wnich are predetermined as "good." That is, the task force felt that the 
evaluation of instructional strategies should be a process which leads to an 
identification and validation of those strategies which are most effective, 
rather than a process which is intended to confirm that certain strategies 
are "good/* in short, the task force felt that research efforts with regard to 
the evaluation of instructional strategies should test assumptions regarding 
relationships between particular teaching behaviors and particular learner 
outcomes. Members of the task force did not view that process as one of 
proving, but rather one of testing. On the other hand, the task force was 
quick to note that research efforts should be directed toward the study of 
instructional strategies assumed to be related to the learner outcomes 
which are deemed most desirable and meaningful. 

Measurement. Problems regarding the measurement of instructional 
strategies were viewed as being enormous, if not overwhelming. The task 
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force recogni;ed the need for better instruments with which to reliably and 
validly mtMsure teaching betuiviors and teaching environments The task 
force ai ^ggesteO that there exists a great need for better instrumen s for 
gathering reliable And valid data with regard to pupil learning outcornes 
Further, it was noted that such data collection procedures needed to be 
more econormcai than is presently the case Problems related to instninien- 
tation were among the most serious noted by the task force, but members 
of the task force seemed to take the position that while it is important that 
educational researchers continue to work on the de>/elopment of better 
instruments with which to measure instructional strategies, they must also 
be satisfied to measure as well as they a>e presently able to measure. That 
IS. researcher* must not be dissuaded from studying certain instructional 
strategies because present mstru . tation is deemed inadequate. The view 
taken was that better measurement .^ill only become possible if researchers 
comnut themselves to measuring as well as they are able those variables 
which are deemed important. It was noted that too often important problems 
are left unresearched because researchers are unwilling to use instruments 
Which do not meet the rigorous criteria usually employed by hardnosed" 
psychometncians in short, the task force took the position that one cannot 
find answers to problems unless one chooses to use the best available tools 
with which to search for such answers. Thus, the task force suggested that 
researchers must continue to improve measurement tools but must also use 
those tools presently available 

Design and Methodology. The task force recognized problems 
associated with research design and methodology to be crucial to the 
evaluation of instructional strategies The task force listed a number of 
problems it saw as being crucial to this dimension. They noted in particular 
that there exists great difficulty m creating designs and procedures which 
isolate those particular instructional strategy variables which influence 
particular pupil learning outcomes. Likewise, they noted that there are 
serious problems associated with creating research designs which will 
allow one to study the relationships between specific teaching behaviors 
and various learner outcomes m a variety of instructional settings. The 
essential issue is the problem of isolating those particular sets of teacher 
attitudinai and behavioral variables — instructional strategies — which are 
related to particular learner outcomes This problem is comp^^nded by 
inadequacies related to instrumentation, research design, research method- 
ology, and statistical analyses. Again, the task force took tne position that 
such problems can only be overcome through efforts which recognize and 
confront these difficulties That is. solutions to these problems are to be 
found in studies which- while recognizing the weaknesses inherent— 
nonetheless utthze such procedures as are presently available. This is not 
to discount the need for greater sophistication. It is simply to suggest that 
efforts in this regard cannot await procedures which are judged completely 
adequately, they must be viewed as tentative and developmental. 

Statistical Analyses. Closely related to problems of measurement, oe- 
Sign and methodology are problems associated with statistical analyses 
The task force recognized that studies which are intended to examine rela- 
tionships between instructional strategies and particular learner outcomes 
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are faced with a multiplicity of problems with regard to statistical analyses. 
However* this is an area m which the task force saw that a great deal of 
recent progress has been made The task force expressed confi<:?ence in the 
ability of statisticians to find increasingly better ways in which to statistically 
isolate important variables. This then was one problem area in which there 
seemed to be a great deal of enthusiasm for present developments. 

Dissenination. The final major problem identified by the task force had 
to do with the need to make research findings more readily available and 
useful to teacher educators and teachers. That is. it was felt strongly that 
research results are too often not available or not useful to practitioners. 
It was felt that research efforts had to be conducted in environments which 
are familiar to classroom teachers and common to normal school situations. 
Further* it was felt that research findings had to be put in a form that class* 
room teachers and teacher educators can understand and internalize. Fur- 
ther, these findings have to be disseminated in ways which make informa- 
tion readily available to practitioners* whether they are teacher educators 
responsible for educating teachers or are teachers seeking to increase their 
own effectiveness. 

Summary. The task force identified six major problems related to the 
evaluation of instructional strategies. Each was viewed as important and as 
a problem needing to be solved. Few solutions were offered by the task 
force. Rather, it suggested that solutions were only to be found in the com* 
mitment of greater effort to what remains one of education's greatest prob- 
lems. 
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1R Priorities and tlie 
lU Nature of Evidence 



AH»n A. Schmhidtr 
Jorto Mark 



The Problttm 

Leading decis'on makers concerned with the directions, structures, and 
support Dases of our American society (of late being joined by more and 
more leaders withm the educational ranksi claim that educators have littte. 
if any. informed exper^ence^-accirmu/afecf eWdence— upon which to base 
decisions about anything that is important in education. Even the teaching 
profession, the g^dup that should have the best firsthand knowledge about 
what works and what doesn't work, is generally m favor of maintaining the 
status quo until more exact evidence" about the educational process is 
available. Educational researchers are scrambling to prove that teaching 
does make a difference and that educators do know some things that are 
not well known to the average person on the Street 

In less than 200 years, the educators of this nation have built what is 
probably the world s most democratic and effective educational system. 
Although the heavens and many other equally inspiring forces were probably 
or the side of this 85 btliion dollar. 60 million participant enterprise, tt neither 
grew like topsy nor evolved without considerable attention to evidence and 
thoughtful decisions. Consequently, it seems ludicrous to charge that to- 
day's educators have no evidence upon which to base program decisions. 
Because of the masS've s^ze of the educational enterprise, the continuity 
of Its existence, and the unusual importance placed upon it by the citizens 
of this Nation, it rs probable that educators are steeped m more evidence 
about the nature of their trade than are practitioners of any other single art. 
trade, or profession in the American social system. 
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\he purposf» at tfus p»ipvf is ru)t U) fitily uppos^l'on to the narrow hard 
evidence twhatevt»r that ibJ wurst^»ppt»f s or thi* educational doomsayers. 
Out to begirt a dtaiosi about hoyfr the profession cun more systematfcaiiy or- 
ganize and use the e^^'st^ng National Storehouse ot Education Mater tais and 
successes that nave tieen so richly and fully Ouftt over the last 200 years 

Some Expected Outcomes 

Short Range 

1 Clear statement of the problem, recommendations about hov^ to 
aeai w^th ii 9»vinq pr.onty to !*^e recommendations, a strategy to 
impiement tne recommendations, and a determination of the next 
steps to be taken 

2 A fif St cut on the nature of the National Storehouse 

3 A t^eqinning wst of locations and persons engaged m major com- 
ponpnt development with a brret description of their programs 

4 A beq'finmq btbiiography of publications relating to the buMdrng and 
use of the NMtiot^ai Storehouse 

5 A f»fst dt.iti ot a puDi-cation presenting the Storehouse case to 
leadir^g educ^tiof s 

6 A specific s* ategy fo.- ♦•.jrther exploring the Storehouse concept 

Long Range 
Pf tmary 

1 Better deftnttton of the levefs of evidence/successes available m 
the Storehouse 

2 Better Knowiedqe of the avaHaonity of evidence/successes at these 
leveis 

3 More systematic rt.iationship between evidence and program/pohcy 

dectsfons 

4 More systematic desipn for uttli2ing and adding to the Storehouse 

5 Greaier use of the National Storehouse 

6 Greater pubitc awareness of tne successes of American education 

; A p^an for more effect»ve sharing of successes within particular 
educational specialities, e g . competency based education 

8 A pian for more effective sharrng of successes between particular 
educational concepts, e g . competency based education and inter- 
action analysts or competency based educat»on and teaching cer^' 
ters 

9 A plan tor more effective sharing of successes between teacher 
education and other profec-s»or>s. e g . competency based teacher 
educatfor^ ^'ind competency based dental education 

10 A plan for more effective snarmg of successes between education 
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.iful tHis.nt-.'. ,iru! rfujustfy r*»q,ir(iing MfTiiK'jr systems, eg . storage 
iinil MMnev.il uf ii»riifH»s infof m.itiofi. Human telat^uns vts a vis the 
communjtv «n educalio" and num.in reiatuins vis a v«s the commij. 
nity »n urban piannmg 

1 1 A plan for rtiore effective shar ,ng of successes t>etween education 
and hus.ness and .ncJustry regaKlmg the relationship between theif 
f espect^ e Storehouses .ind puncy decisjon making 

Sect>n(l»<f V 

1 More effective svstems o* storage and retrieval of validated products 
.ind processes 

^ More effective systen>s of validating products and processes 

3 Better understanding of research and development needs 

4 Less duplication and overlap of research and development efforts 
b More systematic relationship between theory and practice 

b Development of levels of evidence regarding educational success 

The Nature of the National Storehouse 
Wh»t It it 

The National Storehouse is a coMectron of evidence— materials, pro- 
grams. Ideas, concepts, and practices of what works m education today. 
The Storehouse exists What does not exist is an organizational framework 
that could point up gaps m what we know, stimulate efforts to dose them up. 
and provide a typology that woald embrace new. as yet unthought of prob^ 
lems The missmg organiiationai framework would make the evidence 
eastiy accessible to users and would provide information on validation. 

What if Is For 

The National Storehouse would be a user or consumer based index with 
entries classified by topics focused on use or intended users. It would 
therefore be indexed m terms of problems or tasks an educator must cope 
With in order to be effective 

- the practitioner identifies a teacher competency for which he wants to 
provide training, so turns to the National Storehouse for a search of 
what IS available 

- • the pohcy maker is charged with formulating a new piece of legislation 

and needs information about it (e g . reading program), so turns to the 
National Storehouse for research evidence and a bank of knowledge- 
able people 

- the high school teacher of English wants training m how to cope with 
student response to literature, so enters the system seeking English, 
high school, student response— and the level of evidence desired. The 
computer search would identify the appropriate alternative materials 
relating to tne problems as described 
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- the policy maker o nchool board, mteresred m improving its ele- 
mentary stall development program -coutd enter the Storehouse by 
fdentilying school personnel teacher training, elementary school, in- 
service, m order to see if materials exist that address this question. 

In organising the National Storehouse, therefore, we address the question, 
how wilt It assist 

teachers 

teacher educators 

legislators; national policy makers 

researchers 

educational consumers 

educational administrators 

lay administrators 

What It Dott 

The National Storehouse identifies what we have and, as an immediate 
outgrowth, identifies what we need The National Storehouse, therefore, 
shaped by need <or the consumer), identifies for the potential user the 
origins (theory, experience, hunch, invention) of thf- evidence; then it ex* 
plains Its current state of development (idea, pilot test, programmatic test, 
experimental test), its target, its relevance to broad categories of problems 
(policy decisions, research and development programs, practical applica- 
tion in teacher training, and organisation and administration of schooling). 

Organized so it can answer the question. What do we know about X?" 
and open-ended as well, the National Storehouse could classify evidence 
in two ways: 

1 ) by the categories of information a user desires; 

2) different conceptions or categories of educational problems. This 
would permit the user to employ his/her own search strategy and 
also invite him to look at the items in a variety of different ways 

Such an index could be constructed logically, based on some kind of analy- 
sis of problem areas, in a word, the system should be developed deductively 
from what a consensus of practitioners and laymen say we need, not in- 
ductively from what we know we have— although the latter would also be 
included 

Retrl«v«i and Ditten^lnatton 

A user- or needs-oriented National Storehouse must assure easy access 
to practitioners and policy makers. 

The first step toward easy retrieval might be the organization of the index 
of storehouse materials into a simple directory. Contents might be divided 
into: (1) printed materials. (2) audio-visual materials, (3) ideas. (4) people 
and places. Listings should be shorts—product, contact name and address, 
one-line summary, extent of validation, cost. The directory should be set up 
as a loose-leaf binder so that pages may be printed and distributed for 
insert. It should also be free, or the cost should be nominal, covering only 
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ma.l.ng e.pensob Obviously, then, the directory must be funded by Federal 
or foundation moneys This foundation or Federal grant should be large 
enough to pernnt some subsidization of the procucers of the items listed m 
the catalog wh.le producers seek ways to bring down production costs 
Unless materials costs are brought down, the easy access suggestad 
above for practitioner s will not be accomplished 

But to get that directory into the hands of the practitioners will roqu.ro a 
major dissemination program Such a program should be launched by a 
communications expert who is competent to utilize all media approaches to 
the problem. Such approaches would include such standard methods as 
direct man notifications to local and state education agencies, to assoc.a. 
tions and unions, to teacher centers, and to the various national state 
and local professional education associations. But the dissemination pro- 
gram Should go much further. Exposure should m&inly involve public service 
time and space m the education press (national, state, and local) the 
education trade and professional press, local, closed- circuit and cable radio 
and television, local and state conferences, displays at National confer- 
ences. and so on 

The message should be simple. American education is probably the most 
experienced and diverse in the world. And much of what has been learned 
about the way to do things better is available tor educational program plan- 
Costs lor such a program should be explored. A followup to (but also 
simultaneous with) the dissemination program might be the use of trained 
leaders who could go out on request to work with local teachers on use of 
the materials and methods. The communications expert could make suages- 
tions on how best to coordinate this. But unless state and local education 

3"i!;^'~.°L ^ toundation-can underwrite such a program, costs might 
well be prohibitive 

Qu«lity Control 

But a mechanism might be developed to inform users about the qualify 
oi the evidence 

Evidence should be thought of as a device for either hypothesis testing 
or decision making. Evaluation criteria differ, depending on which of these 
two purposes the evidence is to serve. Evidence is generally -harder" for 
hypothesis testing than for decision making, although this may account for 
bad decision making. 

The second factor detei mining the quality of evidence is the stage of 
development of the idea, program, or product. Different evidence is ex- 
pected for a product in its earlier stages than in its -finished" stages Thirdly 
jfirngs that work - hard evidence-w.thin the context of an integrated and 
sequenced program may not work when taken out of that context 

Rather than search for "hard" evidence, the functions of rating might 
better be served by classifying items according to where they are in a 
sequence of steps in the research and development process rather than 
rating them. The sequence might run from armchair ideas, entirely untested 
through ones tried out in laboratory experiments, in simulation or micro- 
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teaching settings, to those field teste i and ready tor dissemination. It 
should be possible lo detine *i set qmte obiectiveiy. and where a piece of 
evidence lies m this sequence should be at ioabt as useful to know as how 
it was rated by somebody if this Kind of rating system was employed, it 
would be possible to enter the Storehouse asking for the level of develop- 
ment (e g . how much the Storehouse had m that subject at a particular 
moment that had been thoroughly field tested) A display could be called 
for Which would show such a quality control process by area, and it would 
be easy to see where weaknesses and strengths lay- a useful tool for 
planning, for commun»cat»ng with legtsiators. etc 

How To Organize II 

A number of models have been proposed, and they are bv no means 
mutually exclusive 

Model A 

I Develop information sources 

1 Gather available materials (including ideas) under tour broad 
headings (a) teaching skill or performance, (b) curriculum materials 
and design by intended outcomes. {O training of educational per- 
sonnel, (d) organi/ation of instruction and schooling. 

2 Prepare for each item a developmental history— who developed 
the product or idea, its theoretical or other sources; its use. its current 
status 

3 Classify each item by the criterion of its effectiveness that the 
user has selected and by other criteria to which it might be relevant. 

4 Gather evidence from the developer and other users about evi- 
dence relative to effectiveness with respect to the criteria. 

5 Sort Items by their relevance to policy, research and development, 
or practice 

II. Develop the tagging system so that anyone who wants to search the 
system can do so. 

1 Invite groups of potential users to ask questions of the system. 

2 Tag items by questions 

3 Identify unanswerable questions 

4 Deternr^ine whether the question is unanswerable because of the 
classification system or because there is no information m (he system. 

Ill Develop validation system 

1 Invite conc^ptualizors of problems to ask questions of the system 
<e g . Jencks asks the system for information on school effectiveness; 
Coleman aFks the system for information on the effects of desegrega- 
tion) 
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2 Invite a group of people to see if they can break" the system 
(e g . asK it Questtons it cannot answer) 

Ultimately, evidence might be categorized by eventual goal and staqe of 
development ^ 



Qoai 



Produce 
Significant 
Study Gams 



First pilot 
idea development te3t 



Field 

TEST 



wide 

USE 



ModefB 

/. Deveiop u$0/user categories: 
Primary User 



Teachers 
Teacher Tramers 
Teacher Trainers 
Decision Makers 
Decision Makers 
Researchers 
Evaluators 
General Public 



Primary Use 



Education of Children at Vauous Levels 

Preservice Teacher Education 

inservice Teacher Education 

Educational Decision Making at Various Levels 

Policy Decision Making at Various Levels 

Heuristics 

Mission Research 

Synthesis 



Each Use or User category would generate its own discrete, though perhaps 
overlapping, set of categories designed to make sense to those interested in 
.t for that purpose (e g.. DiSTAR might be of high interest for elementary 
teachers, moderate interest for teacher trainers, little interest for decision 
makers, no interest for heuristic researchers, moderate interest for mission 
researchers, and be of great interest for synthesizers, and of great use in 
dissemination to the general public). 
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Lonsidar the hardness of data as a relational concept 

TYPE I 

HEURISTIC RESEARCH 



TYPE II 

EDUCATIONAL DECISION MAKING 



TYPE III 

EDUCATIONAL AND POLICY 
DECISION MAKING 



Model C 

1 Seek answers to following questions 

1 What kind of materials should be included'? 

A. Teacher training materials only*' 

B. Nontext materials only? 

C Materials that require trainee to perform? 

2. What IS the purpose of the Storehouse? 

A. To provide the user with a resource for improving his per- 
formance? 

B To supply a research with material t^ empioy m his investiga- 
tion'? 

C To collect information for help in policy decisions'^ 

D. To develop an inventory of materials? 

3. Where should the Storehouse be located'> 
A. A central depository 

B A central depository with regional locations 

C Regional locations 

D No specific location but an office with referral capability only 
4 How would the Storehouse be supported? 

A. Federal support 

B. Federal support and private support 

C. Private support 
D Self*supporting 

E. Other 



ALL DATA 



ROCK 



MELON 



MUSH 



ROCK 
MELON 

MuSm 
ROCK 
MEtON 
MUSH 

ROCK 

MELON 

MUSH 
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How wotiid Storehouse be managed'^ 
A By teachers college types, and research types'' 
B By NEA, AACTE. AERA. ASCD. AFT. ATE. a?id other orgamia- 
tions*? 

C. Other ways'? 

What categories of organ i2at ions will allow the user to enter the 
Storehouse'' 

A Category i — Levels ot Evidence by Results 

1 Paper and pencil test 

2 Verbat(2eor writ& what one learned 

3 Perfornn m a simulated situation 

4 Perform in a classroom situation 

5 Short-term results with students 

6 Lasting (long>term) results with students 

7 Other 

B Category 1 1— Subject Areas 

1 English 

2 Math 

3 Other (including Special Education and Early Childhood) 

4 Generic 

C Category ill— Teacher Activities 

1 Planning 

2 Questioning 
3. Diagnosing 
4 Many more 

D. Category IV— Setting 

1 Urban 

2 Rural 
3. Other 

E Category V— Grade Levels 

1. Primary 

2. Intermediate 

3 Other 

F Other Categories 
Content of each entry 

A. Description of material 

B. Cost 

C Where available 

D Time for use 

E Source and nature of development 

Materials should be ' keyed ' to other big questions of: 

A. School finance 

B. School policy 

C. Social concerns 

D. Other 
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CLASSROOM 
PRACTITIONERS 

NONCLASSROOM 
PRACTITIONERS 



Dissemination use 



DEVELOPMENT USE 



RESEARCH USE 



POLICY USE 



AFFECTIVE 
EVIDENCE 



TESTIMONIAL 
EVIDENCE 



STATISTICAL 
EVIDENCE 



HISTORICAL 
EVIDENCE 



NO 

EVIDENCE 




A MANY SOME NO 

THEORY THEORY THEORY THEORY 
BASE BASES BASES BASE 
The figure illustrates a 120-ceH cube which depicts a multiple category 
system which would be used to analyze, sort, or describe Storehouse ma- 
terials. 
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The camgorv system would have thr^e dimensions -^one set of five 
descrtptors of types of evidence, one set of four descriptors of theoretical 
bases, and one set of descriptors of user categories The last would be 
subdivided mto multiple user-determmed subcategories 

EvMenee Oeterlpiori 

The descriptors related to evidence deal with the type of evidence that 
supports the material They are 
Mas Affective Evidence 
I like It 

It had a positive effect on me 
It was lousy * 

Has Testimonial Evidence 
We did It and it works ' 
It had no visible effect ' 
Only three teachers are now using it ' 

Has Statistical Evidence 

A Statistical random sample 
0001 

Has Historical (Judgmental. Philosophical. Theoretical Evidence 
Accord»ng to NiE 
For 100 years 

Has No Evidence 

0 

Theoretical Calegorlet 

It IS conceivable that if one could name the various theories currently 
supporting teacher training material, one could establish a set of descriptors 
accordingly Until such is possible, a four-part category system might be 
usable 

Has a Theoretical Base Has Some Theoretical Base 

Has Theoretical Bases Has No Theoretical Base 

UMr Caiegof «ee 

These categories are basically undetermined They will name as the 
descfiptois those which will be used to extract information and material 
from the Storehouse. The categories will be determined by surveying of a 
particular group of users as to What kind of information or material they 

need*>' "What questions will they ask of the Storehouse What 7 * 

While any category of User * could form a group to be surveyed, at least sik 
can be readily identified 

1. Policymakers 

2 Research users 

3 Developer users 

4 Dissemination users 

5. Nonclassroom practitioners 
6 Classroom practitioners 
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Siibf9Ct 

Trainmg educa> 
tional personnel 

Teaching students 



Programs 

administrators 
counselors 
teacher ti 

reading 

independence study 
value clarification 



evidence 

microteachmg 
protocols 



lesson units that work 
and specify pupil outcome 
at particular level 



Usor concerns 

M ) clinical concerns 

2) research 

materials development 

notions and ideas about 
education 
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Possible use 



Evidence 



Criteria Control Matrix 



Policy 



Has 



Research and 
Development 



Practitioners 



None 



Theory 
Support 



Sound 
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None 



Model F 

Th# National StorthcuM of Educational Mat<»fial»/SueeaMM: 
EvWene* Needad tor Poltoy Miking/Program BuHdIng 

• Define Hard Evidence (Class i Evidence) 

• Estimate Demand/Need for Hard Evidence 

• Estimate Availability of Hard Evidence 

• Estimate Gaps Between Demand and Availability 

• Estimate Cost and Time to Fill Gaps 

• Recommend Alternative Strategies for Filling Gaps 

• Def.fte Less Hard Evidence (Class ii Evidence) 

• Etc. (Repeat steps above until supply and demand are equal) 

• The Need to Educate the Public about 
--What evidence is 

--What IS available 

—What evidence professional educators will use and why. i.e.. wo 
explain and tell them, they don't lay it on ust 
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Somt S«l«ct»tf E»«mpt«t of National Slorthoutt Componanlt 

Tb» N«tk)nai StorvhouB* of Education*! MMtrMi/Suecwttt- 
T»««5h«r Etfuestlon CompoMnt~Stu0l«t of Tvwrwr E}tt«tiv«n«»» 

1896 Characteristics ol the best teachers as recogni/bd by students 

^Kl A\Z I 



Pirst quantitat.vfi studies relating cortipetence ot teachers to 
various other characteristics iMenami 



1905 

1930 An analysis of 209 different systems for rating teachers (Barr 
and Ev^nu^ 

- 1945.46 intensive analysis of teacher s role m classroor^i (Anderson 
et dl) 

1948 Survey Of investigations .nto the measurement and predictions 
of teacher effectiveness (Barn 
79 Studies analyzed 

46 used supervisor s or principal s ratings 
15 used grades m student teaching 
18 used measure of changes m pupils 

1951 Assessment of socio-emotional climate m learning situations 
(WithalJ) 

7 categories of teacher statements 
1954 Identification and review of all major teacher effectiveness 
studies completed between 1900 and 1962 (fWorsh and WHder) 
900 references 
360 abstractions 

20 predictors of effectiveness identified 

1958 Analysis of 1.000 studies on teacher effectiveness (f^it^ei and 
Cross! 

20 of the studies (2%) involved actual measures of 
teacher effectiveness 

I960 Analysis o( teacher characteristics (Ryans) 

6.000 teachers m 1 700 schools m 450 school systems 
12 general 'trends ' 

1966 Analysis Of verbal behavior in the classroom (Beilack) 
4 pedagogical moves 

4 functionally different types of meaning communicated 
by teachers and students 

5 general roles for classroom language game 

1971.73 Search for validated teacher characteristics or competencies 
(Rosenshine and Fursti 
SO studies 

1 1 mam characteristics 
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1972 Analysis of «itudent learmnq (Jencks) 

Most tosis used meti<iure student learning 
msens'ltve to differences m teaching behavior 

1960*74 Evaluative teaching units 

Flanders social studies/ anthnr^etic 
Schauta and Nuthaii ~ electricity 
Joyce social studies 
Noy literature 

1974 Exhaustive review and assessment of studies of teacher ef- 
fects on students (Dunkm and Biddlei 

Th« Nuttontfl Stor^houM of E(fuc«tlon«l Maltrlait: R^fer^ncM 

Teacher Training Products The Study of the Field <750l.* Gage 

Protocol Materials Catalog ( 1 12). Florida SEA 

A General Catalog of Teaching Skills. Turner 

A Catalogue of Growth in the Pedagogical Domain. Hudgms 

The Houston Module Bank (225K University of Houston 

NiE Research and Development Center Products (est 100) 

Fai West Laboratory Mimcourses {?0\. Rerlmer. Borg. Flanders. Ward 

Validated Teacher Characteristics or Competencies/Process Variables 
(11). Rosensh no and F ur st 

Critical Elements in Teaching (42). Cru»ckshank 

Sharing Educational Successes— E SEA Title I 

Sharing Educational Successes — ESE A Title ill 

Tested Modules m Competency Based Education (Googol). 120 CBE iHE s 

Data on Teaching and Learning m Theses and Dissertations in Education 
and Related Fields (Coogol*piex) 

•Nu<T^O«^ •'^ pd^entr>e%e^ ln(llcat^(»^ aopfommaf^ number o* ptodu^'t* mdudeo 

The National SlorehouM ot Educational Materlalt/Succauat: 
Taacher Etfucallon ComponanI— Soma Baglnning RmHfncM 

Barr. A S The Measurement and Prediction of Teaching Efficiency A 
Summary of Investigations. Journal ot ExpenntontBl Education, XVI 
(1948). 203^283 

Cooper. James E A Survey of Protocol Materials Evaluation, Tampa. Flor- 
ida Leadership Trammg institute m Teaching. University of South Florida. 
1974 

Oomas. S J . and Tiedeman. D V "Teacher Competence An Annotated 
Bibliography. Journal of Experimental Education, XIX (1950). I0l-2l8 
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Gacje N i et ai Compuferued Catalogue of Teacher Training Products 
.^•fSHiifi^h .ma OrvPiopfTutnt Momfjf.mcJum No 116 Palo Alto California 
Stanfurd Cerwvr tor K.rs^^rr.h ami Development m Teach.ng St^inford 
Univer^,itv 1974 

Mudqms Brvce B A Catalogue of Cor^cepts m the Pedagogicai Domain 
h\ lou.s M.bsouf. vVabh.ngtofi umvers.ty Graduate institute of Educa- 
t»on 19/4 

Gwaehnes for Delivery of Devefopment Products ( Draft i Washmgton 
D C N.ftionai mst»tuteof Education 1974 

Protocoi C^t^iiog Matet.ais for Teacher Educat-on Tallahassee. Fionda 
State Department of Education. 1974 

Schm.eder Alien A. and Joyce. Bruce The National Storehouse (for CBEi 
two versions fuii and abridged Washington. DC USOE Office of 
Career Education. Education Systems Development. 1974 (Draft) 

Schm,eaer Alien A and Mark. Jon. ed? The Nat^on^I Storehouse of Educa^ 
tn)na! f^aterjats/Successes Washington. D C USOE. Office of Career 
tducat.on. Education Systems Deveiopmunt. 1974 (Draft) 

Shanng Education Success A HandDook for Validation of Educational Prac- 
tices Washington. D C United States Office of Eoucat-on. National 
Advisory CouHCM on Supplementary Centers and Services. 1974 

Turner. Riuuito L .ed ) 4 General Catalog of Teaching Skills 

The National Storehouse of Educational Materlalfi/Succesgee: 
Profile of Some Potential Experience Bases 

1 16.500 school districts 

700 community* schools 

325 schools using individualized instruction 

4.500 teaching centers 

X alternative schools 

2 50 states 

20CBE mandates 

9 career education mandates 

4 teacher center mandates 

3 1 200 teacher trairmg institutions 

120CBE programs 
600 teacher centers 

The National Storehouse of Educationai Materlale/Succesees: 
Some Interesting Facts 

« Many powerful educational policy makers do not think that .t exists 
> Many educators feel that .t is already too bountiful and needs no fur- 
ther money for d*»velopment 
J Many feel that ERiC has the Storehouse well under control 
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4. Storehouse matef i«ih with less validatron sell faster. 

5. Most new validated materials are etther on a shelf somewhere or used 
»n a Single classroom or program 

6. Most consumers do not like to go to some other place to use cata- 
logs and materials banks/resources— they want them located m their 
office. 

The Newsletter is one of the highest demand consumer items in the 
dissemination business. 

8. There is a very thin market for many of the most important materials 
developed. 

9. Regarding product validation—what might be hard evidence to one 
kind of consumer is useless to another kind! 

10. Many validated approaches are born in theses and dissertations— and 
eventually die there 

11. Most producers are reluctant to share materials (a) which are un- 
validated; (b) which might be plagiarized or copyrighted by someone 
else 

1 2. Publishers are reluctant to produce high risk materials. 

13 AH producers are potential consumers and all consumer*' are potential 
producers. 

14. College and university staff generally feel that they are the only ones 
sufficiently qualified to produce valid materials for widespread use- 
school staff generally teel that college and university staff generally 
don't know enough about the real world to produce valid materials. 

15. Materials m the process of "becoming" are usually more related to 

up front" research and development people than those that have 
already become*'— yet dissemination strategies almost always relate 
to "finished products " 
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1^ The Politics of 

If Teacher Competence 



H«rv«y B. Scrlbner 
L»Oti«rd B. St«vtnt 

Since the matter of politics generally is not raised in most di'iCussions 
of teacher competence, the title of this paper perhaps requires a brief 
explanation. -Politjcs" as used here does not refer to the partisan activities 
of political parties or the influence that political officeholders may have 
occasionally or regularly in issues of education. Rather, -poJitics*' refers to 
the activities— which, though nonpartisan, are keyed to identifiable interests 
and goals and thus are, m the broad sense of the word, political activities— 
and pursuits of groups and individuals within the arena of education. 

American public education is well known for its unpolitical character. 
Clearly, neither the public schools nor the public universities developed in 
a political or social vacuum. It is equally clear that there has been a notable 
and commendable absence of overt political, partisfin control of the nation's 
systems of public education. 

From this, however, one should not assume that educational institutions 
function without a political atmosphere. They do have and always have had 
a political atmosphere. Local school boards, for example, typically pride 
themselves on their independence from city nail. Vol all but a handful of 
school boards are dependent on political officeholders for raising and appro- 
priating public tax funds for schools. Moreover, every school board to some 
extent pailicipatas in the deployment of '/iscal and human resources for 
reasons that more than occasionally strotch into political or quaslpolitical 
considerations. Similarly, while state departments of education typically 
work hard to develop an Insulation from statehouse influence, rare indeed is 
the chief school officer who does not know his way around the halls where 
statutes are written and appropriatirns are made. Nor can those who man- 
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age the public universities be accused of total innocence of the art of politics. 
Oespite the academy's characteristic disdain for politics, university heads 
with budgets and construction programs m mmd find it necessary to do 
periodic fence-mending with elected representatives of the public who con- 
trol the strings io the public purse, inside the academy more than a small 
amount of enerov if expended on political maneuvering. 

In brief, the public education enter prise, despite its unpolitical reputation. 
IS quite political when politics ' is defined in broad terms. Schools and 
universities affect, and are affected by, external political forces. In addition, 
education systems, because they are large organizations with important 
social functions and immense resources, possess a set of internal political 
dynamics which at ttmes become quite intense. Teachers and professors 
organize for special ends, and they are not disinclined to display their 
-JOllective power to achieve their ends. Administrators are no different, 
except that they have the advantage of power which comes from their posi- 
tion. Likewise, school boards and university trustees have political power, 
and they use it; in addition, they band together in organizations on the 
regional and national levels, and these organizations themselves wield a 
certain amount of influence over the nation's ed. cation systems. Even 
researchers, who often are regarded as the most objective and least politi- 
cal of ail inhabitants of the education forest, clearly have their own values 
and distinctive perceptions of what is and what ought to be. And to the 
extent that researchers channel and shape their research and their conclu* 
sions in keeping with their value system, they, too, participate in education 
politics. "Man is by nature a political animal,*' Aristotle told us long before 
politics acquired its undeserved, distasteful connotation. And while he 
perhaps did not have educators in mind, the observation clearly did not 
exclude them, as contemporary history of schools and universities has 
made quite evident. 

There was a report in the New York Times^ recently that two public agen* 
cies in New York City, the Board of Education and the Board of Higher 
Education, had met together for the purpose of establishing closer coordi- 
nation of their respective activities and programs. Now, since both agencies 
have responsibilities in public education and since both spend public dol* 
lars, the effort to coordinate surely is commendable, and the New York 
Tlm&s was quick to do precisely this a day or so later on its editorial page. 
As a kind of footnote to the story, it was reported that once the heads of the 
two agencies had met, the next thing they did was to meet privately with the 
heads of unionized teachers and faculty members. 

The New York Times, it would seem, should have commended the agen* 
cies not only for their positive instinct to coordinate, but also for their keen 
sense of pragmatism. 

This is not necessarily to condemn this "touching base** syndrome, though 
one is left to wonder precisely what private understandings were reached 
by the agency heads and the staff unions and to wonder as well whether 
any understandings were reached that might impair the coordination effort 
in some way. In any event, this incident is a clear example of politics at 
work. The point is worth making that political activity should be recognized 
as such by those who participate in it as well as by those who observe it. 
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It is neither statesmanship, as some seem to view it: nor is it an Indis- 
pensable element in modern public administration, as others sometimes 
contend. 

The «ie ot teacher competence, like other policy questions in education, 
has political overtones. Anyone who reviews the professional literature on 
competence and performance in teaching is struck first by the volume of 
writing on the topic in recent months. The EducBtion fndsx for one recent 
12-month period^ makes reference to more than 230 articles on the educa- 
tion of teachers— no small number of them pertai directly or indirectly 
to the question of performance or competency. The second striking char- 
acteristic of the literature Is the influence peddling that currently is going on. 

Perhaps "influence peddling" is too strong a way of putting it. Perhaps 
one should say that the literature abounds with the work of individuals 
contributing to a controversial debate. Whatever one wishes to call it, 
there is no lack of information on where the major constituencies stand in 
the debate vis ^ vi$ the political questions of a) who is to control the edu- 
cation and licensing of teachers and b) how is competence to be defined. 
Lines continue to be drawn that make clear who is the t}ehaviortst and who 
IS the humanist; who is the teacher educator and who is the teacher em- 
ployer; and who is the professional spokesman and who is the public advo- 
cate. 

One might conclude that education needs a Dr. Kissinger to work out a 
settlement and achieve a truce — but perhaps not. Perhaps what is needed, 
before the issue can be settled with any degree of finality, is a set of guide- 
lines for working toward a settlement. Given the present state of the public 
schools and the educational concerns of the society to which the schools 
belong, it would seem that those with a role in the design of policies and 
programs for teacher competency should keep at least the following general 
points in mind. 

First one should be aware of special interests, not disguise or ignore 
them. The policy designer need not necessarily make moral judgments as 
to "good guys** and '*bad guys.** but he ought to be aware of the politics 
of policy. Since policy affects people, people will exert all the influence at 
their individual and collective commend to shape and control policy in its 
formative stages so that emerging rsolicy will be in their best interest. 
Needless to say. the goals of a special interest do not necessarily make fot 
good public policy or for better schools. Often, indeed, the goals of special 
interests ignore the public interest. thou£*> the rhetoric of interest groups 
might have the public believe otherwise. 

In the formulation of performance systems for preparing and licensing 
teachers, it ceems dear that teacher educators, who have long controlled 
teacher preparation and thus heavily influenced teacher certification, tend 
to favor quite strongly the retaining of their favored position. This does not 
mean they are against systems that ostensibly will enhance teaching per- 
formance. But it does mean that they probably will have a very special 
interest in retaining as much control as possible over teacher preparation 
programs. The teacher educator may typically be biased toward a belief 
that the center of gravity of teacher education belongs on the university 
campus. In contrast to the view of many school administrators who favor 
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school based teacher training 

Similarly, the longstanding objective of organized teachers to control 
the teaching profession, much as attorneys and medical doctors control 
their professions, .s readily apparent ii. the current debate over teacher 
performance Organized teachers, m desiring to professionalize teaching, 
argue that controlling the gates to the piofession is essential to the making 
of teaching .nto a true profession. Whether the organized teacher s affilia- 
tion IS with the National Education Association or the American Federation 
of Teachers makes httle difference: the desire to control the profession 
transcends the matter of aMination. One president of the NEA. sometimes 
viewed as less aggressive than the AFT. said a few years ago that the NEA 
has long dreamed c: controli.ng -who enters, who stays in. and who leaves 
the profession.- Further, he went on. "Once this is done, we can also con. 
troi the teacher training institutions."' it seems unlikely that th- expressed 
desire would be rejected by teachers who choose to affiliate w m i.-ie AFL- 

The policy designer clearly should be informed. He should be cognizant 
of the varying views on teacher competence, and he should not write off 
views on the ground that they emanate from a special interest organization 
But he should consider the source-which is to say that he owes it to the 
public to consider the motivation beneath the position. The views of or- 
ganized teachers, teacher educators, or school boards obviously need 
to be considered. But so. too. should hidden political agendas be identi. 
fied. and the policy designer should be surprised when he finds no ulterior 
motivation at work. 

Second, one should be aware that public education is beyond the era of 
seemingly unlimited expansion and is well into a period, indefinite in length 
of flattened curves. A teacher shortage has turned into an overabundance 
of teachers. Public dollars for education, whatever the governmental 
source, are diminishing Student enrollments are likewise flattening out 
and decreasing m some instances. 

Clearly, there is meaning in this development for those interested in 
teacher competency systems. School boards are interested more than ever 
m teacher producfwity. They are turning, also, for the first time in recent 
memory, to the disemployment of teachers. In more than scattered cases 
teachers are being laid off much like airline pilots and automobile factory 
workers for lack of funds, fewer students, or a combination of the two The 
policy question therefore arises— if performance or competence is to be 
the standard for employment, should it not also be the standard for disem- 
ployment? For if not. are not schools open to the charge of dual standards 
—performance as the standard for employment, and "last hired— first fired" 
as the standard for dismissal? Needless to -.ay. there will be sharp disagree- 
ment as to the proper solution. Organized teachers will probably argue 
hard that where some teachers are to be dismissed, those with longest ser- 
vice deserve the most protection, it should be equally needless to note that 
senio.'ity has little, if anything, to do with performance, and thus this argu- 
ment essenti.=»lly is an antiperformance stance. There is. of course, another 
side to the coin. A school board intent on saving dollars and with no con- 
cern for the rights of teachers will quickly conclude that the greatest savings 
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are to be made when the most senior teachers— those on highest salaries — 
are dismissed, as opposed to younger teachers on lower salaries. This 
stance, too, is antiperformance: therefore, the question remains concerning 
how to shape performance systems that apply both to the entry of teachers 
into employment and to the involuntary exit of teachers from employment. 

Third, in the design of competency systems, one should keep in mind the 
growing mood of the public for *• stronger hand in the governance of 
schools. More than a few public school parents are correctly acting like 
consumers. They are asking for accountability systems. They are demand- 
ing effectiveness in educational programs. They want choices for their 
children. In New York City and Detroit, and in other cities as well, parents 
are deeply interested in the decentralization of school power. In the case of 
individual schoois. parents are demanding closer involvement in decisions 
that affect school management and the spending of tax dollars. However 
much some spokesmen for organized teachers may dislike the idea of 
parents coming into schools to view the performance of teachers* and 
however difficult parent involvement may make life for school principals 
and school superintendents, the fact remains that parents are getting jus* 
tifiably tougher m their demand to be involved and increasingly savvy in 
pursuing this objective. 

It seems that performance systems for preparing and licensing teachers 
must take account of the likelihood that parents likely will move increasingly 
into school decision making. In New York City, for example, parents elect 
community school boards, they have a specific role m the employment of 
school principals, they have the right by policy to be consulted in the hiring 
of community school superintendents, and they have one small foot in the 
door in tenure decisions.^ Most important, perhaps, they have an estab- 
lished channel for appealing grievances against local decisions to the high* 
est level of the city school system, and the short history of the grievance 
process has shown that parents have scored more than a few points by 
taking their Case to higher authority. 

Given such precedents, it would seem reasonable to speculate that 
parents will want and will insist upon a growing role in the assessment of 
teacher performance. Yet in the current clobate over competency systems, 
while there is frequent concern for the desirability of participatory planning 
by organized professionals, school boards, state authorities, and teacher 
educators, there is little, if any. consistent concern for also including 
organized parents m the planning Parents, admittedly, are less cohesively 
organized and thus less easy to identify as representative than the other 
major parties to the debate. But ignoring the valid interest of parents in the 
systems that ultimately will emerge is. perhaps, to cou^t a serious problem. 
For if parents are not involved in the planning of teacher competency sys- 
tems, especially those that deal with the matter of licensing, then the 
possibility IS more than remote that parents will tend to view the new sys- 
tems with more than passing skepticism. In brief, would it not be both 
ironic and tragic if policies are developed for the purpose of promoting 
higher competency in teaching through the systematic involvement of all 
interested parties except parents, whose children are to be the direct benefi- 
ciaries of better teaching'' 
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Fourth, teuchm t.<irup<»tent:y systems should be designee' m the knowl- 
edge that the Amencdn .,chool however resistant to reform it is ani* 
however sunaar «t is regardless of geographical location is nonetheless an 
institution that is changing and that is characterized by local diversity. 

If performance systems are to be levers for school reform — and they can 
and should be they should be designed on the assumptions that school 
reform will accelerate over time and that teacher performance is subject 
to varyng definitions m keeping with local priorities and neecs. There are 
such thir^gs as alternat.ve schools, and their number is on thv^ increase.* 
There is the distinct possibility that public education will come to adopt the 
notion of public schools of choice. ' Responsible efforts are underway to 
e>ipand external diploma opportunities for high school students and external 
degree programs for college students. Expansion of various forms of 
learning-by- doing and learmng-through-expenence -whether in the form 
of unconventional apprenticeships or more conventional work-study pro- 
grams seems likely in the coming years. The very definition of "teacher- 
's in a process of evolution as more and more adults who ere not profes- 
sionally tiamed. full-time educators enter the classroom in the form of 
parent volunteers and occasional instructors of specific subject matter, 
from home buying to creative writing. All these trends are altering, slowly 
but inexoratDiy. the meaning of education, the boundary of school, and the 
definition and role of teacher These trends should be of critical interest 
to those who undertake the highly complex task of stating what teacher 
competence is. how it is lo be systematically created, and the measures of 
competence. For it would seem self-evident that as schools are changed, 
so. too. will the competencies needed for effective teaching be altered. 

School reform moves with halting steps Schools -whether in Utah or 
New York. Minnesota or Texas— tend to look very much alike. But status 
quo and sameness are not the whole story. If the movement for teacher 
competency takes as given the assumptions that schools will remain in a 
relatively unchanged form over time and that teacher competency will be 
defined in relatively the same way despite locality, then the movement, it 
would seem, is based on two very dubious, uninspiring assumptions. 

Fifth, and last, one should keep in mind the central purpose of compe- 
tency systems, wh.ch is to improve the educational lot of the young. The 
point IS obvious, and all will readily agree with rhetorical, politically benign 
statements that competency based teacher education is not an end in 
Itself and that the ultimate objective is ' the improved delivery of educa- 
tional services 

Delivering on the promise, however, will be quite another matter, and one 
which wilt require policies with teeth. The clear implication of compe- 
tency systems is that competency at some pomt. in some fashion, wilt be 
assessed, and not only will performance be assessed, but where it is found 
wanting, something will be denied to someone in the name of pubiic pro- 
tection from sub«-par performance The task for the policy designer is a 
difficult one in one sense, it is to ensure that the policy which emerges is 
not so acceptable to all interested pafties—teachers and teacher educa- 
tors in particular --that 'the improved delivery of educational services" is 
less than a certain result 
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What, then, should be the gufdehnes for developing teacher competency 
programs and policies? 

The first, it would seerri, should be to strive for flexibility Teacher compe- 
tency systems should allow for multiple definitions of competence anr* 
multiple methods of assessment Clearly, teacher competence has one set 
of concrete meanings for Chicano famUies who want for their children a 
formal education that is bicultural and bilingual. Competence may have 
other sets of meanings for black parents in central Hariem, for the middle 
class families of Shaker Heights, or for the blue^coilar families in the mining 
areas of Pennsylvania and Kentucky. This is not to speak Indirectly in favor 
of dual standards of expectation for children, nor to encourage the con> 
tinuatton of ghetto schools, nor to deny that some competencies will have 
universal application, it is merely to say th'it different localities will demand 
some different competencies from their teachers>-and that competency 
siystems should not only tolerate this diversity but encourage it. 

A second guideline should be an inherent capacity for self*reform. Unless 
competency systems are to be a new orthodoxy as impervious to change as 
fhi* course-accounting system of teacher education which Dr. Conant found 
to be bankrupt more than 10 years ago,* yet which still is with us. mecha- 
nisms for review and renewal should be built into the new performance 
based systems 

A third guideline relates to the earlier discussion of school reform. It 
(s hoped that the designers of competency systems will make the value 
judgment that learning opportunities are enhanced dramatically to the 
extent that formal school programs recruit and use artists, writers, crafts- 
men, and other skilled adults in the teaching of the young on a regular, 
systematic basis. The issue, in brief, is whether competency systems can 
and will encourage a broader definition of who may teach, or whether they 
will encourage schools to remain inflexible m the use of human resources. 

A final guideline deals with the matter of control Teacher education, it 
has been said, *'is m a unique bind, caught as it is between the forces of 
public control, academic control, and control by the public schools. 
While this is true, it should be added that the bind has been characterized 
by rather strict separations*— the universities controlling the preservice 
education of teachers, the state agencies administering the licensing ap- 
paratus on the basis of academic credits and attained experience, and the 
public schools operating inservice programs often consisting more of 
shadow than substance. The proposed competency systems, with shared 
planning and perhaps shared control, too. include the prospect, as some 
observers have noted, of closer ties among teacher educators, school sys- 
tems, and organized teachers. Few. it has been said, "will argue with the 
desirability of this goal 

If what is meant by this is that teacher education, thanks to competency 
systems, will be more rational and more connected to the nation's public 
school classrooms, then few should argue. But perhaps there is room for a 
question if the result of these new ties is to be the inevitable thrusting of 
near-complete control of teacher education into the hands of professional 
educators, whether at the university level or the level of the classroom, or 
a combination thereof. One of the fundamental problems with public educa- 
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tion. one tends to cofn:iu(n». .s its over dependence on professionals. One 
recalls the observation of Dr Conanl. who certainly was not a merTiber of 
the New Left, that what goes on m schools and colleges is far too impor- 
tant to be left entirely to the educators 

Clearly, there .s need m public education for professionals with higher 
levels of performance and higher prote^sional status Exclusive control of 
the teaching profession, however, .s tTiore than a professional matter, .t is 
also a matter of public policy The case often is made by teachers that they 
are professionals as much as doctors and lawyers, and therefore they should 
have comparable professional power to regulate and control their own 
ranks This argument, however, selectively ignores two central factors: 
a) teachers are public employers and b) thoir clients are m the classroom 
by law and not by voluntary choice. These are more than incidental pcmtS; 
indeed, they constitute powerful arguments for greater public control of 
teacher education and for greater parent participation m the planning and 
operation of teacher competency systems. To raise this point is not to say 
that teachers have no legitimate interest in shaping teacher education and 
teacher compewncy systems it is to say that the public has an even larger 
more fundamental interest in these matters because of the obvious facts 
that the schools are public agencies and the young are compulsory students 
Yet. obvious as they are. these points tend to get lose m the intricate debate 
as to what precisely n is that constitutes competence in teaching and how 
exactly this competence can and should be measured 

Indeed, the answers to these questions— what constitutes competence, 
and how is competence to be measured— will reflect the value system of 
the individual or the group to whom the question is put. This, m essence, is 
the politics of teacher competence. And this is the reason that the parent 
deserves as much or more of a role m the design of programs and policies 
for teacher competence as the professional 
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WThe Decision -Making 
Process in Planning 
and implementing 
Education Programs 
that Meet the Needs 
of Our Diverse Society 



Carl A. Grant 



This paper on decs-on making consists of three general sections a 
rationale for educational decs.on making, a climcal model of action and 
a report from a national survey It is written to prov.de the reader with a 
Clearer understanding of the issues and problems of planning and imple- 
meniing a CBTE program that reflects and responds to the multicultural 
population of our socety 
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Rationale 

The United States is a nation whose citi/ens have constantly dos-red and 
oemcnded the r-ght to full participation .n the decs.on-making processes 
Which influence their way of life in 1776. the original thirteen colonies 
rebelled because they were forced to submit to taxation by the mother 
country. While being demed any meaningful representation ,n those gov- 
ernmeniai bodies whicf. r.ad the power to decide both the extent of the 
taxes to be levied and the uses to wh.ch the collected taxes would bo put 
in 1955. m Montgomery. Alabama. Rosa Parks igmtod a social and econom- 
■c revolution by not acqu.escng to those laws and regulations- legislated 
and enforced by others-thai denied her the right to participate fully m 
American society 

There IS similarity between these iwo histo ical elements Both events 
call attention to the desire and demand of American citizens to fully par- 
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tictpatD in the decisions that mtluonce an6 have impact on their way of 
lite Both Ovents a^e also enampies of tho strategies used Americans to 
alter the decision-makmg processes which deny individuals equal represen- 
tation and. therefore, equal participation m their government. 

The history of education cannot be eKammed separately from other his- 
toncai events, smce education is an integral part of the American culture 
Today the educational system is experiencing challenges comparable to 
those issued m 1776 and 1955 People representing various role groups — 
students, teachers associations, and the economically and socially disen- 
franchised- are demanding their right to fuUy participate tn making those 
decisions which govern or influence their lives. Full participation implies 
parity -the equal partnership of diverse role groups and cultural groups 
sharing in the making of decisions which affect their education and their 
social and personal growth 

The cbdiienges such a partnership poses for the educational community 
are formidable and long overdue but should prove to be a rewarding en* 
deavor The necessity for such a partnership in education decision making 
»& addressed m the final report of the Higher Education Task Force on 
Improvement and Reform m American Education (HETFiRE) The report 
states that 

We. the Task Force, believe that partnership in teacher 
education, manifested m different ways m various activities. 
IS the Key element m educational reform through teacher 
education individuals are paramount, and what education 
does for and with larger society wdi be a result of decisions 
made by individuals interacting if one accepts this thesis — 
that social action ts a product of individuals interacting m 
groups — the degree to which education responds to social 
needs will Oepend largely on the extent to which individuals, 
working together cooperatively, are motivated toward a com- 
mon goal * 

The report further adds 

that educational policy at whatever level must be based 
on the needs of the related community Local assessment of 
educational needs should determine local educational policy 
just as the assessment of the educational needs of the nation 
should determine Federal education policy. 

The requirement that schools become more responsive to 
local needs indicates that it must be the local needs that 
direct what the schools should do— that policy governing 
the decisions made and implemented m a school is made at 
the local level. This requires that policy made at levels in- 
creasingly more remote from learner and school (at the 
district, state, and Federal levels) must be the product of, 
rather than the prescription for. local educational policy. This 
notion requires that policy-making "begins" at the local level. 
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With poltcv madi* hy moro general puOfics betng made to 
tacihtate rather than to constrain the tleveiopment and im- 
piem^ntat.on of local policy Good pol.cy^rDakmg that 
leading to reform m the mterest of learners- has the follow- 
ing prerequisites 

1 Those to be involved must have organised a policy mak- 
Tig body representing all of the constituencies to be 
affected The Josh Force has named this the Policy 
Board 

2 What should be done must be determined by all to be 
involved 

3 Decisions regarding criteria for determining the effec- 
tiveness of management and operation must involve all 
parties 

4 Commitments to shared responsibility m ongoing evalua- 
tion must have been established 

5 Agreements on qualification criteria must have been 
established for selecting and retaining those who will 
implement policy 

HETFIRE IS only one of many educational groups to recognize and attempt 
to respond to the present educational needs of equal representation and 
equal participation m education decision making. Within the last 2 years. 
therA have been a significant number of publications in journals and a 
s.gnificant numbe. of conferences which have treated educational reform 
governance, and collaboration By and large, this treatment has been more 
exhortation than action Power is not given; it must be taken. It cannot be 
assumed that because we hold conferences and write articles about gov- 
ernance, parity, and collaboration .n regard to educational decision making 
that these concepts will be implemented A case m point is a phrase m the 
Declaration of independence, nationally adopted almost two hundred years 
ago The following words have still not come true for all Americans: 

We hold these truths to be self-evident— that all men are 
created equal, that they are endcvod by their Creator with 
certain inalienable rights; that among these are life, liberty, 
and the pursuit of happiness. 

in any hone*;t smcere discussion of the decision^makmg process m edu- 
cation, attention must also be given to another pomt-^those who participate 
in the decision making process must represent the racial and cultural diver^ 
sitv of this country. It cannot be taken for granted that just because various 
role groups are represented, that the racially and culturally diverse popula^ 
tions of this country are also represented for most Often they are not. 
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Most of the types ot role groups which have been historically identified as 
necessary components of any educational decision making body— college 
administrators. Federal and slate officials, leaders in teacher organizations 
—are those who have been most inaccessible to nonwhite, non-Anglo> 
Saxon people. In addition, decision making bodies in education have not 
included all typt^s of role groups, community representatives constitute one 
such group that has largely been ignored. 

Clinical Model of Action 

What follows is an outline of a workshop that can be utilized as a method 
of heipif^g educators to better understand the actual process of planning and 
implementing CBTE programs that respect multicultural society. The four- 
part model IS based on the premise that actual participation in the process 
of decision making is as necessary as theory and good intentions. 

Pant: The Selling 

After being given the rationale for the clinic (see preceding section), 
the participants will be given a scenario of a college undergoing change. 
The Scenario is designed to provide the participants with one possible 
example of a college attempting to embark on a new multicultural CBTE 
program. The scenario also provides clinic participa.'its with a number of 
character roles that can be used for actual practice in decision making. 



SCENAR!0 

State Coftege is a multipurpose institution in terms of the diversity of its 
program of studies and the diversity of its students. It is located in an urban, 
cosmopolitan city with a heavily diverse population of Anglo-Saxons, Blacks. 
Orientals, and Puerto-Ricans. Since the Brown decision in 1954, State Col* 
lege has dedicated itself to meeting the needs of the community by making 
its educational programs available to all people regardless of race, color, 
creed, sex, or national origin. It is a 4-year degree -granting institution with 
master s degrees being offered in Elementary Education, Early Childhood 
Eaucation, and Social Work. The total educational program is organized 
into 10 divisions, one of which includes the Division of Teacher Education. 

':e*^cUeT Education. This division trains approximately one-third of the 
total number of graduates of the college. The students are provided with 
theory and practice necessary to help them become '^proficient'' elementary 
or secondary school teachers. 

The Division of Education includes six departments: 

Early Childhood Education 
Elementary Education 
Secondary Education 
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Health and Physical Education ** 

Audiovisual and Educational Television 

educational Disabilities 

Each den^rtmeiu ,s headed by a department chairperson, and the total divi- 
sion is headed by a division chairperson who is a recent addition to the 
faculty, recommended highly by the dean of the colleges academic affairs 
for his/her competence in affecting change. 

I. -.s;, actional Program. Though the graduates of the Division of Educa- 
t,on have been certified and highly recommended in some area schools 
there is general dissatisfaction with the teacher education program and it's 
relevance to the social, economic, educational, and cultural aspects of the 
commumty. The educational program is noted for its competent instruction 
m four or five curricular areas. The students are exposed to proficient 
lecturers and adequate multimedia. The program is designed around 
separate sub,ect areas, usually charactered by a series of three credit- 
hour courses with autonomous instructors who focus on their own compe- 
tence and interest. These curriculum areas are based on Independent goals 
and objectives. There are innovative practices such as microteaching 
mteraction analysis,, a clinical approach to supervision, independent study 
opportunities, an inner-city tutoring program, and a bilingual tutoring pro- 
gram. The majority of the academic program is campus centered. 

Though these innovative practices and traditional elements are yielding 
positive results, they never fit together in a way which gives maximum pay- 
off to the students or the community. Thus, the division chairperson and 
the Elementary Education Department chairpa.-son decided to look for ways 
to put the pieces together into a program design based on careful planning 
the immediate and relevant needs of culturally diverse student teachers 
and a multicultural school population. They decided that some of their basic 
assumptions about teacher education, previously based on tradition, needed 
rethinking m relation to teachers' roles and to what and how we want chil- 
dren to be and to become. After observing several teacher education pro- 
grams and many models of CBTE. they decided that one way was to begin 
looking at teacher behaviors, attitudes, and understandings that facilitate 
pupil growth physically, emotiondiiy. and culturally. 

It is evident that State College is involved in a stage of educational re- 
form. The success of this reform effort will depend upon the Inclusion of 
diverse role groups and cultural groups: it will also depend on the Inter- 
relationship of those elements essential for planning and implementing an 
educational program that meets the needs of a multicultural society 



Part II: Planning the Program 

After the clinic participants have had an opportunity to read the Scenario 
they will be instructed to select volunteers to come before the group and to 
portray roles either identified in the Scenario or suggested by it. which 
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would be necessary and/of helpful m the inittal planning of the program at 
State College. The dime coordinator is trying to determine whether the 
role players will choose to plan their program changes using only the 
characters associated with the State College or whether they will choose to 
reach out and involve other community members (i e.. lay individuals and/or 
groups, school district personnel, labor groups, or possibly state and 
Federal officials) The coordinator will also be observing whether or not the 
participants address any of the following questions: 

1 Who constituted the program staff established to study the feasibility 
of developing the program? Were minority group members on this 
staff 

2 Should an educational needs assessment be made? What educa- 
tional needs should be considered in planning the program? Was an 
assessment of minority educational needs made? 

3 What ideas or suggestions were solicited concerning multicultural 
education'^ Which individuals or groups wore asked about this? 

4. Should the program seek to change or influence institutional goals 
and/or objectives'^ Should goals and objectives relate to the prob- 
lems of cultural diversity'^ 

Thtf /ules that the oarticipants select and the issues they address in the 
mitia? planning for the program will be listed. The coordinator will not 
provide feedback at this time 



Part ill: Implementing the Program 

The participants are now ready to start implementing a muUicultural 
CBTE program as suggested by the Scenario and as initially planned (see 
Part I). The role playing will be continued and the participants will be 
allowed to add or delete roles depending on what they learned from the 
planning stage or from what they expect they wilt need for implementing 
the program 

Again, what is primarily being looked for here by the coordinator is 
Choice of roles and the issues they address concerning the implementation 
of the program. Some of the questions the players might raise are the 
following: 

1 What criteria should be used for admission and retention of students 
in the program? is there provision for recruitment of minority stu- 
dents'? 

2. Are teacher competencies specified which relate to understanding 
cultural diversity'^ Are attitudtnal changes and modifications of 
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teacher behavior included in the CBTE program? Should minority 
group members be involved in determining what attitudinal changes 
are needed? 

3. What roles should students and faculty have in decision making'^ 
Are there provisions for staff development m relating CBTE to needs 
of multicultural education? 

4. Who Should select and design instructional materials? Are instruc- 
tional materials identified according to some criteria relative to 
multicultural education? 

5 What provisions should be made for continuous feedback of informa- 
tion from all participants in the new program to the assessing agent 
and/or program staff? What plans can be made for program modifi- 
cations based on feedback? What provisions can be made for feed- 
back and change from minority group members? 

The roles adopted by the participants ana the issues fh«y address m 
implementing the program are again listed. No feedback is provided at this 
time. 



Part IV: Small Group Di«cu«»ion 

Since only those who have participated in the role playing will have had an 
active part in the dmic up to this point, all those attending the clinic will 
now be mvited to jom the proceedings. This will be accomplished by having 
at least one of the role players join a group of not more than six other 
individuals. Each of these mimgroups will then be given an adequate amount 
of time to further refine the issues that were raised by the role players 
regarding the planning and implementation of the multicultural CBTE pro- 
gram. A brief oral report of these findings will then be made by a spokes- 
person from each of the groups to the entire gathering. 



Part V: Feedback and Evaluation 

The roles adopted and the issues raised by the participants in Part 1 1 and 
Part III. as well as the additional input provided by the minigroups in Part 
IV, will be summarized by the clinic coordinator. The coordinator will 
compare and evaluate the results of the clinic with the findings of a recently 
completed national survey of CBTE programs. 



Report on Survey Results 

Very little research hcs been done to assess the effectiveness of CBTE 
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pfograms or to determine ttu» char»ictef istics of des»qn. plar^nmg. and 
jmpiementation which result m that effectiveness This paucity of research 
has rrteant that those involved with the development and conducting of the 
programs are forced to operate »n the dark, with few empiricallv tested 
guidelines to assure that their programs will succeed m attaining the 
desired ends To remedy this situation, it is imperative that measures of 
various program characteristics be developed and examined m the light of 
their effect upon educational outcomes. 

One of the few studies so far condviCted toward this end is a survey con- 
ducted by Grant and Calhoun (1974). wli.ch investigated conditions m 8? 
colleges aces*: America wn«c.'« are m the process of setting up or con- 
ducting CBTE programs One major purpose of this survey was to 
examine characteristics of these colleges m relation to their effect upon 
the planning and implementation of the programs and their approaches to 
multicultural education. 

Toward this goal, the Grant and Calhoun study employed rigorously con- 
structed scales to measure the amount of input mto the piograms from 
students. mino"tv groups, and the community at large, the pluralistic content 
of the Curriculum responsiveness to minority needs, the amount of attention 
devoted to careful planning, and the degree to which administrators per* 
ceivedthei' programs as instruments of educational change. One of the more 
striking findings of this research emerged from a factor analysis of the 
mtercorrelations among these instruments— all of which loaded rather 
heavily on a single factor This result indicates a strong degree of cohesive- 
ness among the program characteristics, wherein a program which is 
strong m any one of the assessed areas will probably be strong m ail ol thn 
other areas as well The factor underlying this cohesiveness was tentatively 
interpreted as quality of program. 

When the componAntc of this factor were examined in the context of 
various characteristics of the institutions supporting the program, such as 
Size, selectivity of admissions policies, library Size, the ratio of Ph.D 's on 
the faculty, etc . it was discovered that there is virtually no relationship 
between the success of the program and the quality of the supporting insti- 
tution This finding seems to imply that careful attention to the interrelated 
variables which determine the quality of a CBTE program can result m the 
development of a good program at any college, almost regardless of the 
overall finances and facilities of the college. 

Within the intercorrelations among the scales, it is noteworthy that one 
particularly strong relationship was found. The extent to which m'ncrstv 
group members have input into the program is correlated (r ■• .76) with 
the adequacy of the program in meeting minority needs. This relationship is 
of such magnitude as to suggest that institutions whose programs do not 
receive input from minority group members will not provide a program ade- 
quate to the needs of these peopie. 

In addition to these findings, tabulations of responses to several other 
Items in the questionnaire provided results of interest to the present dis- 
cussion 

1 A large number of the colleges and universities surveyed did not 
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pst.-iWish any cntw,.. Un (J«;term.mng who would participate m the 
•nitiijl planning 

2 Over a third of the institutions reported no community involvement 
.n planning moit aspects of the CBTE program Those community 
representatives who tooK part (Ikj not include many mir^onty mem- 
C>ers 

3 in a large percentage of the programs, minority group members were 
not involved in determining proposed attHudmal changes and modi- 
ficauOf\s of teacher h*>havior 

4 Only one-half of the institutions have provisions for the recruitment 
of minority group students into the CBTE program 

5 The majority of the available instructional materials were not 
Identified according to criteria relative to multicultural education. 

e«l^^<^"" ^T"""- ^'"^y represents only an 

exploratory venture mto the sea of open questions which surrounds CBTE 
Extensive further investigation is an absolute necessity .f educators are 
ever to conduct teacher education with any confidence that they will arrive 
at the goals they set out to reach 



Summary 

Planning and implementing an educational program that reflects and 
responds to the multicultural population of our society would be an exciting 
Challenging endeavor, indeed, .f fh.s idea could be realized, we would have 
taken an affirmative step toward rectifying a system which now denies many 
of our citizens their fundamental nght to representation m any decision, 
making process that seriously affects the quality of 'heir i,ves Racism is 
stili serving as the keeper of the gate, consequently most programs sweep 
concepts such as parity and pluralism under the institutional rug. It will 
take courageous and dedicated individuals to implement such a program 
because of the overt and covert racism that permeates every aspect of our 
society NoMetheless. this does not mean that such a program should not 
fJQ attempted and cannot be developed 

Finally, it .s interesting to note, and is worth reporting again that one 
significant finding of the study is that .f educational programs are to 
respond to the needs of minority grriup people, the programs must have 
input from minority group people If we are serious about havmg a piogram 
that responds to our diverse society, we must not fool ourselves and think 
we can get by with a token individual or two: we cannot. To reform an edu- 
cational program to meet the challenges and needs of our diyerse society 
we must have individuals frorr different racial and role groups fully partici- 
pating in the decision-mak.ng process. To reject this idea is to further 
perpetuate racism 
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Some Unscholarly 
Views on 

Institutional Change 



Don Davft» 



The competency based idea has become a movement, tt has alt the stan- 
dard Characteristics*— confusion, proliferation of ideas and activities dif- 
fering schools of thought, lots of energy and talent, strong advocates, and 
outspoken critics. I'm pleased that the idea has "taken off." However, 
there is one central issue about the direction of the movement that should 
be addressed. 

The question is this: Will competency based teacher education remain 
largely a research and development effort, focused largely on changing 
teacher education programs m colleges and universities, or will it move into 
the larger arena: teacher competence, the range of things represented by 
the slogan "accountability." the measurement of success of schools, the 
involvement of the community in educational decision-making, and the 
evaluation of professional personnel? 

The first direction is relatively manageable and relatively noncontrover* 
sial. The chances of succeeding are relatively good— if this »s the difArtion 
chosen. But. the narrowness of the objective means that the enterprise will 
t)e relatively instgntficant. The path of highest potential success is also the 
path to triviality. 

If the second direction is chosen, the competency based movement will 
suddenly be center stage and mainstream activity surrounded by extra- 
ordinarily complex political and controversial problems and issues. This 
direction is obviously the significant one. the one in which activities could 
have profound impact, but it is also a direction in which chances of success 
(or to be more precise, of achieving institutional changes) are very slim 
indeed. 
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I cannot pfesimie to toll yow wtiat you shoufd do ( do not even teoi con. 
fident about pred.tlmy wh<u will happen I simply want to call the dilemma 
to your attention 

What I will try to do now is to mention a few points that seem to me to be 
relevant to your concerns about competency education and institutional 
change What follows is a collage ot thoughts, not an orderly and systematic 
analysis or synthesis and certainly not a theory of change 

(11 The first point is a question Who cares "> Those of us involved in 
teacher education as a major part of our professional lives have a difficult 
time accepting the fact that people outside of our relatively small group of 
career professionals don t care much about teacher education or "educa. 
tional personnel development, if you prefer a broader term This field of 
study and work has a very low priority on most university cafr>puses. in most 
school systems, m state governments, in Congress and the iedurdl educa- 
tional agencies, among the large professional organizations, and among the 
public at large The topic is considered either bonng or not very important 
My own experiences in Washington over the past decade or so confirm the 
truth of this assertion Decision makers in the agencies and on the Hill were 
concerned about the teacher shortage • and were willing to support legis- 
lation and appropriations that addressed that problem When it became 
Clear to nearly everyone that the teacher shortage had become a teacher 
surplus, we faced a very difficult task winning support for appropriations 
under the Education Professions Development Act. We often heard "If 
there is a teacher surplus, why should the Federal government be spendmg 
money for teacher training- which increases the surplus*? " Our persistent 
efforts to Show that most of the funds were actually for inservice staff 
development anc* that the mam purposes had to do with the improvement of 
the quality of education in schools and colleges never really penetrated the 
barriers of skepticisms about teacher training. " The widely held view that 
training doesn t make a difference was reinforced by the growing view that 
neither teachers nor schools make much difference 

What I am saying should be borne in mmd you dream about financial 
support for research and development m competency based teacher edu- 
cation It doesn t seem likely that massive amounts of money for this 
purpose will be made available Excessive ambitions for support will lead to 
more frustration i remember some reports in the past 2 or 3 years that 
talked about investing more than $300 million m "teaching complexes." 
$80 million or more in protocol materials." and tens and hundreds of 
millions m other related research and development activities. That's carry- 
ing wishful thinking to the point of absurdity, given the general low rank of 
educational personnel development m the minds of people who make deci- 
sions about budgets 

(2) The second pnmt is that changing individuals through framing will 
not produce mstitutional change. Individual change is a necessary but not 
sufficient part of institutional or systemic change. This is a lesson that we 
learned through the experience of the NDEA and EPDA institute and fellow- 
ship programs. Helping individual teachers and administrators gain new 
knowledge, attitudes, and skills is essential but not very productive if the 
person with new learning is placed back in an unchanged system. The 
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system will ysu^iMy wm it w«ih this lesson th<)t led us to develop a broader 
concept Of training m the Urban/Rural School Development program and 
the ilUfated Educational Renewal plan 

(3) The third point is related to the second One of my great hopes 
for the competency based teacher education fdea at the outset (way back 
when the Bureau of Research took the initiative to fund the elementary 
models proiectsi was lhat the professional aspects of preparation would 
move largely from the campus to the field M seemed logical that a program 
emphasizing competencies and competency development would be 
primarily field based, and would force the development of new and more 
effective relationships between training institutions and the schools and 
communities m which the graduates of those institutions serve 

The training complex idea was one significant model it envisioned the 
creation of a neutral field training site to which both the university and the 
school system would contribute and from which both would draw The 
problem was that the training complex idea required important institutional 
changes on the part of both the university and the school system. Both 
institutions resisted and very little really happened, except for a few very 
prumising pilot projects The incentives that were offered were not suffi- 
ciently strong to create the necessary changes m institutional relationships 
and the power allocations 

Whether or not my hope that PBTE would lead to having most teacher 
preparation take place in the field is beginning to be realized I'm not sure: 
I'm not even sure whether that hope is widely shared I continue to hope 
that It 1$. 

(4) The fourth point is that the individual school is the most manageable 
and appropriate unit for institutional Change in education. John Ooodlad 
has been making this pomt for a long time now. and my experiences over 
the past few vears support his view One of the basic ideas m the Educa* 
t«onat Renewal plan was that the locus of planning, change, and reform 
should be the individual school and its community If the idea of pluralistic 
local school problem solving ever really caught on, there would be a vast 
new market for the kinds of competency based materials and training pro- 
grams that are being developed 

(5) The fifth point derives from my Washington experience The question 
IS How Will the results of research and development and evaluation be 
used? (This IS really a general question, not one specifically related tc 
research and development in competency education.) This is a straw-man 
question, of course, because i believe that decision makers don't often use 
the results of research, development, evaluation, policy analysis, and other 
academic endeavors. This is a more polite way of saying that the decision* 
making system intends to be irrational, influenced primarily by political, 
personal, and pragmatic forces Better research or better analysis will not 
correct the deficiencies of the system any more than training teachers 
better will Change school systems, in my most cynical moods I am willing 
to say that l cannot identify one ma|or and profound decision made by the 
Office of Education, by the various planning and budget offices, by the 
Congress that was honestly based on the results of research or program 
evaluation I can cite many decisions that were made on other grounds — 
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partisan politics, oducaiionai politics, personal bias, desire to save money, 
ani« on and on. My point is that those concerned about research and devel- 
opment and evaluation m education must also be concerned about en. 
couragmg. developing, and working for ways to mal<e governmental and 
institutional decsion-making processes more rational-more retponsive 
to information, ideas, and documented experience 

{6> The next point, which has some relationship to the last, is that we 
haven't made much progress yet on the problems that Research and De- 
velopment people m education have m communicating effectively with 
either policy makers or with practitioners. NIE is having more than Its share 
of difficulty wit^ this problem, just as I did when i was on the firing.line. 
I can t pretend to know all of the solutions to the problem. I certainly haven't 
been successful in solving it myself However, one of the first things to be 
done IS to clean up the language— to communicate in plain and powerful 
English. A second step towards solution is to recognise that most people 
in educational research still seem to be bound to a top-down program based 
on the familiar model-research, diffuse, demonstrate, install, evaluate, 
revise, and begin the process anew More than we admit, we tend to believe 
or hope that there is a passive institution waiting to receive and adopt the 
results of our Research and Development endeavors. Obviously, this is a 
faulty model of institutional change 

(7) Another point, which I mention only briefly, is that very high priority 
Ought to be given by the Research and D<»veiopment community in educa- 
tion to studying the process of implementation of change, with special 
references to the complex matters of incentives. For example, it is dear 
that the incentives for initially adopting an innovation and the incentives 
for implementing it— making it work— are v«ry different 

(8) And now. a final point is the question; Who runs the schools? The 
question should actually be. Who runs the society? The most importam 
issues we face as a people today have to do with the relationships between 
government, the institutions of the society, and the people. And it s a ques- 
tion that has a lot to do with institutional change. Too many people see 
institutional change as a technical and professional task. Too many people 
see institutional change as the responsibility of the social scientists, the 
educational research and development people, the innovative bureau- 
crats in Washington, or the cutting edge" superintendents Too many 
people have come to accept the elitist idea that ordinary people cannot be 
trusted to make deciaions in a complex and technologically sophisticated 
society because they can t possibly understand the consequences of their 
decisions^ Hence, we need the "best and brightest" to make decisions— 
always, of course, m the best interests of the people or the "national secu- 
rity.' I believe we should fight this elitist self-fulfilling prophecy. 

It is my view that our most important task as a people Is to bring our 
institutions closer to the people, to make those institutions more responsive 
to the diverse needs and interests of the people, and to find ways whereby 
ordinary people can share decision-making powers with the professionals 
and experts m various fields. The work that i am doing now is focused almost 
entirely on this task in relation to dti^en participation m decision making in 
the schools. It is dear to me that schod improvement, school change. 
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school reform, institutional change in educatton will happen to the extent 
that we learn how to involve parents and the community directly and sig* 
nificantly with teachers and administrators in a process of planning and 
problem solving, particularly at the level of the individual school. Without 
this kind of parttctpatory process, change jsn*t likely to occur (or if it occurs, 
it isn't likely to last), and what is even more important, it shouldn't occur 
Despite all the concerns and warnings expressed in these hurried com^ 
ments. i believe that educational research and development people and the 
competency education movement have a good deal to contribute to this 
kind of process of institutional change 
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Task Force on 
Institutional Change 




JamM Oy«r 

This task force consisted of from 12 to 15 memtjers (at different times) 
and decided at the outset to discuss and develop the research issue as a 
Single group. The chairman. Dell Felder, introduced the game play and 
suggested that the context be broadened to include common issues that 
influence change in all sizes and types of institutions. 

It appeared that two assumptions were made generally by the group 
(though not actually verbalized until much later). One was that the insti- 
tution that was referred to was. in fact, the institution of higher education, 
and the second, that the change referred to was the implementation of 
CBTE. 

Introduction around the group disclosed a variety of representatives 
accounting for all critical change interest groups with the exception of the 
professional teacher organizations. The brainstorming technique was used 
to bring forth a flood of issues, primary areas of interest being increased 
costs, professional autonomy, administration involvement, and the general 
relationship currently existing between the university, public schools, pro- 
fessional teacher organizations, the community, and the state departments. 
(Note: these will be herein referred to as the groups.) 

There were strong feelings about the magnitude of change to be instituted 
in the initial stages, the commitment of the various authority figures, and the 
fact that change, as such, could never be "rammed down anyone's throat." 
Many of the group continued to clamor for more proof" before change, 
though others recognized the need for establishing valid research in actual 
experimental programs before such proof could be forthcoming. 

The recommendations for research issues and their group-assigned 
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priorities are as follows 

1. What, specifically, can IHE & other groups learn from each other? 

2. Should research efforts be shared by the five groups? 

3. What are the prerequisites for collaboration between all five groups? 

4. Is a new professional school of education desirable? 

5. What are the emerging functions of professional schools of 
education? 

6. Are pioneer (pilot) programs desirable? 

7. How do you develop a pioneer (pilot) program to influence the entire 
system? 

8. How is a successful prototype program implemented as the pro- 
gram? 

9. How do (job) supply and demand influence institutional change? 
10. How can threat to faculty and professional autonomy be avoided? 

1 1 What rationale is there for nonchange in all five groups? 

12 Is the traditional reward system really a major influence on change? 

13. What does institutional change include? 

14. Do institutional histories and culture make a difference in develop- 
ing change? 

15. Do educators t)elieve teacher education needs to be changed? 

16. Can the Teacher Education Center help develop institutional 
change? 

1 7. Should change (CBTE) be limited to undergraduate areas? 
Feeling was Strong that the research personnel should include the 

interested parties, and that all prior related studies be consolidated and 
promulgated as soon as possible. 
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Model for 
Developing a State 
System for Managing 
Performance 
Programs* 



RolMrt C. Burithart 



Overview 

The results of a study completed by Dr. Robert C Burkhart. President of 
Educational Assessment Systems Corporation, and supported by the New 
York State Education Department indicate that student teaching evaluation 
forms in use in many New York State institutions of higher education with 
approved programs m education consist largely of judgments made on the 
basis of either subjective requirements or subjective criteria The results 
also imply that the evaluation of student teachers emphasizes portions of 
professional education for which formal instruction -s seldom provided. 

By supporting this study, the Division of Teacher Education and Certifi- 
cation of the New York State Education Department was attempting to 
gauge the present state of the evaluation of student teachers in approved 
programs m education, it was felt that publication of the results of the study 
would allow teacher preparation institutions to measure their own progress 
toward the implementation of a competency based teacher education pro- 
gram 

Conclusions drawn from the results of the Burkhart study indicated a 
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significant contrast t>etween a ma|Ority of the student teaching evaluation 
forms now in use and the assumptions and principles outlined in **Tho Edu- 
cational Rights of Students." a statement prepared by Burkhart and Mike 
Van Ryn. Chief of the Bureau of Inservice Education, Division of TeachAr 
Education and Certification. NYSED (see Table 21.1). 

To conduct this study, data were gathered from the student teacher eval- 
uation forms used in 80 New York State institutions of higher education with 
approved programs in education The data were analyzed to determine the 
emphasis placed upon various portions of student teaching and the assess- 
ment standards employed in making judgmeniS about the performance of 
the student teachers. A result of the analysis: a ' Representative College 
Student Teacher Evaluation Form * {see Table 21.2), a prototype of the 
forms now in use. was constructed and analyzed. 



The Study 

The 89 collected forms were analyzed to determine the total number of 
discrete evaluations required by all forms. A total of 3.191 discrete items 
were identified and 3.015 were coded according to the Educational Assess- 
ment Systems Corporation Analysis System. 1 76 items were discarded be- 
cause they were too general to be classified. 

The EASC Analysis System is a description in list form of the functions 
required of a student teacher (see Tab!e 21.3). For the purposes of this 
study the instrument was presumed to be exhaustive. The Analysis System 
consists of three major areas: The Teacher-Learner Pre- Instructional Func- 
tions Area, the Teacher-Learner Instructional Functions Area, and the 
Teacher-Learner Professional Profile Area. The areas are divided into cate- 
gories as follows: Pre-instructional Functions Area, four categories; in- 
structional Functions Area, four categories; and the Professional Profile 
Area, two categories. The ten categories are further divided into a total of 
57 subcategories. Each subcategory has its own distinctive number. The dis- 
tinct number of the subcategory was the number used to code the discrete 
evaluation items discovered in the 89 evaluation forms. 

The decisions regarding classification were made by individuals who 
had received an orientation in the use of the Analysis System. Each item 
was rated separately by more than one judge and the inter-rater reliability 
factor was found to exceed .90. 

To construct the "Representative College Student Teacher Evaluation 
Form* (REF). it was necessary to complete two independent analyses of 
3.015 items. As a group, the items, coded according to the subcategories 
in the Analysis System, were tabulated to determine their frequency of use 
in the 89 forms. While number 23 of the subcategories was not mentioned, 
the remaining 56 subcategories showed a frequency ranging from 1 to 500 
times on the 89 forms. 

In the second analysis, each of the 3.015 items was examined to deter- 
mine the assessment standard employed. The following is a list of the stan- 
dards found to have been used, the code identification, an explanation and 
example of each, and the percentage of the total number of items repre- 
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Standard 


Code 


Explanation/Example 


Per Cent 


Category 


C 


The name of an area in which an 
assessment is to be made: * Ques- 
tioning." 


14% 


Definition 


D 


The name of an area in which an 
assessment is to be made and 
mention of the distinguishing char- 
acteristics of the < rea: "Subject 
matter comprehension: the under- 
standing of COnceotfi thrnunh 
teaching.** 


1% 


Subjective 
Requirement 


SR 


A statement that requires a sub- 
jective judgment about achieve- 
ment: "Effective use of black- 
board.** 


72% 


Subjective 
Criteria 


SC 


A statement that requires a judg- 
ment about a level of achievement: 
{scale required) Acceptance of 

COnStruCtivA CritiCicm ** 


7% 


Performance 
Requirement 


PR 


A statement that requires rating 
based on behavioral occurrences: 
**Accurate reports handed in on 
time.** 


6% 



Based on the two analyses. 50 actual evaluation items were selected to 
constitute the REF. All subcategories that had appeared at least once on 
more than 60 percent of the 89 forms were represented according to their 
relative frequency. The 50 REF items also reflected the percentages deter- 
mined by the second analysis. For example, "subjective requirement" was 
found to have been employed in 72 percent of all evaluations; therefore. 
72 percent of the REF items employed a subjective requirement as an as^ 
sessment standard. 
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Figure 21.1. below, graphicaily represents the processes involveri in the 
study, as described above, from the collection of the student teaching 
evaluation forms to the construction of the REF. 



Figure 21.1 



Step 1 



89 student teaching evaluation 
forms collected 



Step 2 



Step 3 



3.191 discrete items identified 



3.015 Items coded according to 
EASC Analysis System 



176 items 
discarded 



Step 4 




3.015 items analyzed 
to determine frequency 
on 89 forms 



3.015 items analyzed 
to determine assessment 
standard employed 



Steps 




REF constructed 50 items selected 
to reflect frequency and assessment 
standard analyses 



The following charts highlight the primary findings of the Burkhart study. 
Chart 21.1 — Emphasis on Areas of Assessments in REF Determined by 
Frequency of Item Occurrence— presents information abou- the emphasis 
placed on various portions of the REF. The number of evaluations per sub- 
category and category and the percent of the whole represented by these 
figures by category and area are presented. Chart 21.2— Emphasis on 
"Subjective Requirement" and "Subjective Criteria*' Assessment Standards 
in REF Determined by Frequency of Use— presents the number of evalua- 
tions in one category that employed either a "subjective requirement" or 
'subjective criteria." The percent of evaluations in one area that employed 
either a subjective requirement or subjective criteria has also been cal- 
culated. 
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Chart 21.1 

EMPHASES ON AREAS OF ASSESSMENTS IN REF 
DETERMINED BY FREQUENCY OF ITEM OCCURRENCE 



il 

E 

^ 


>» 

? j5 >» 1^ ^ *n irt 

^il 51 1 ^1. U 
i^ll ||J5 lis §15 ill SI 

2^-^ in III III III is 


Preasso$sm9nt of 
Pupil SHiH and 
Techniques by 
Teacher- Learner 


1 2 


1 


1 


2% 




Teacher- 
Learner 
Pre* instruc* 
tionai 
Functions 


Pfanntng insiruc- 
tion Objectives 
and Strategies 


2 1 


1 


1 


< \i 


10% 


Setting Up Pro- 
cedural f^OUliHAS 
for Instruction 


3-1 
3 2 


2 
1 


3 


6% 


Maintenance of a 
Positive Learning 
Environment 


5-1 
5-2 
5-3 
5.4 


1 
2 
1 
1 


5 


10% 


46% 


Teacher- 
Learner 
Instructional 
Functions 


Conducting 
Learning 
Exper lences 


6-1 
6.3 
6-4 


4 

1 

3 


6 


16% 


Knowledge of 
Content Area and 
Procedural SkrUs 
Used During in- 
struction 


7-3 
7-4 


5 
4 


9 


18% 


Assessment 
feedback - 
Remediai Help 


82 


1 


1 


2% 


Assessment of 
Continuing 
Professional 
Development 


9.3 


1 


1 


2% 


44% 


Teacher - 
Learner 
Professional 
Profile 


Professional 
Characteristics 
and Interests of a 
Teacher 


10.1 
10-2 
10-3 


6 
5 

10 


21 


42% 


TOTAL 


ia 


50 I 


50 


^ 

100% 100% 
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Chan 21.2 

EMPHASIS ON SUBJECTIVE REQUIREMENT AND 
SUBJECTIVE CRITERIA- ASSESSMENT STANDARDS 
IN REF DETERMINED BY FREQUENCY OF USE 
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it 
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Pr^aft»»tftnnom of 

Pupil SJciii and 


t 




1 


1 


1 00% 




Teacher* 
Learrter 

Pfe«lntlruC> 

tionai 

r unction* 


Planning inft'uC* 
And Stf AtffQios 


? 1 


1 


1 


1 


1 00«.o 


60% 


S«tting up Pro- 
cedural Rou 

for in%fruct(0n 


3-1 
3 2 


;? 
1 


3 


1 


33% 




M«int<»nance ot a 
PCfttttv« Learning 
Environtt^ent 


5' 1 
6-2 
53 
5-4 


1 

2 
1 
1 


t 


3 


60% 






ConOuCting 
Loarning 
Eaper (Onces 


6-1 
64 


4 

1 

3 


6 


5 


62 S% 




Teacher- 
Learner 

instructional 
unctions 


Kno^iedQa ot 
Content Ar^a and 

Procedural 5i«iis 

Uted During 
m»|ruCt«on 


73 
7.4 


5 

4 


9 


5 


56 5% 


60 9% 


Astessment 
fa^bach - 
Remedial htaip 


82 


1 


1 


1 


100% 






A$sei,$mont ot 
Continuing 
Ppofe«fttondi 
Development 


93 


1 


1 


1 


100% 


100% 


Teacher. 


Profeft»ionai 
Character ifttici 
and intetettft nt 
a Teacher 


10.1 
10.2 
10-3 


6 
& 
10 


21 


21 


100% 


Learner 

Profeft&ional 
Profile 


TOTAL 1 


18 


50 


50 


39 
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Although the major conclusions are largely self-evident, the most impor* 
lant results should be mentioned and briefly discussed because of their 
implications 

Chart 21.1 illustrates two major findings: (1) little emphasis is placed on 
the pre-instructional functions of most student teachers and (2) consfder- 
able emphasis is placed on the professional characteristics of the student 
teacher While an almost equal amount of weight is given to the instruc- 
tional Functions Area and the Professional Profile Area, this is so only be- 
cause one of the categories in the Professional Profile Area, Category 10. 
• Professional Characteristics and Interests of a Teacher," constitutes 42 
percent of all the evaluations. That is, while the evaluations for the other 
areas are distributed throughout the categories constituting the area, the 
evaluations »n the Professional Profile Area are almosc wholly located within 
one category. Furthermore, the number of evaluations in one subcategory, 
number 10-3, is greater than the number of evaluations in any other single 
category, it is also of interest tp note that there are no evaluations from 
Category 4, 'Occupational Responsibilities " 

Chart 21.2 demonstrates that the bulk of evaluations employ either "sub- 
jective requirements" or "subjective criteria" as assessment standards. 
There Is only one category, number 3, "Setting up Procedural Routines For 
Instruction,** m which less than 50 percent of the evaluations do not employ 
subjective assessment standards. All the evaluations, or 100 percent in the 
Professional Profile Area employ subjective standards. 

When the information presented on Charts 21.1 and 21.2 is compared, 
the conclusions are significant All evaluations in the Professional Profile 
Area, which constitute 44 percent of the evaluations, employ subjective 
assessment standards. The one category that does not use subjective as- 
sessment standards for more than one-half of Its evaluations constitutes 
only 6 percent of all the evaluations made. While the conclusions are 
obvious, their prominence should not belittle their importance. Most student 
teachers are being evaluated by predominantly subjective assessment stan- 
dards in all areas. Furthermore, the largest single area of evaluation, which 
relates to professional characteristics, is wholly evaluated by subjective 
standards, it is not, therefore, extreme to conclude that most of the student 
teacher evaluation forms used in New York State institutions of higher edu- 
cation with approved programs in education are not framed to insure the 
application of consistent criteria, the making of fair judgments, or the ac- 
curate reflection of the educational programs of the institution. * 

As teacher education practices in New York State become competency 
based, it will be important to assure that the evaluation of students be con- 
sistent With instruction and based upon explicit, publicly stated standards. 
It is hoped that this summary of the Burkhart study wiM be helpful to the 
institutions of higher education in New York State as they assess their 
efforts in converting to competency based education programs. 



'A Cursory revitfw of tho eoucatior) courMft otfftrad by 46 mstitui>or)» of higher dducafion 
m Nttw York State that offer approved program* m educatior) located no couraes that vyere 
Wholly devoted to Educational Professionalism* and found only mfrequani mention of thai 
or related topics <n courfie descriptions 
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TMm 21.1 

THE EDUCATIONAL BIGHTS OF STUDENTS ENROLLED 
IN TEACHER EDUCATION PROGRAMS 

Trie educationrJ rights cited t>eiow should be viewed within the context 
of a democratic society that guarantees all citizens certain human and civil 
rights. Moreover, they should be considered as congruent with those rights 
held by faculty and institutions and consistent with the principle of public 
disclosure. 

The educational rights of students rest on the following assumptions: 

1) that achievement evaluation is only justifiable m areas where instruc* 
tion has been provided; 

2) that, when instruction is provided, evaluation is essential; 

3) that an instructional system, if it is to be responsible, must concern 
Itself with the explicit basis for instruction, provide feedback, and pub- 
licly disclose the reouirements to be met; and, 

4) that evaluation should be consistent with instruction and be congruent 
with the explicit diagnosis of students as indicated by their profiles as 
learners. 

Therefore, in a teacher education program, the student has the right to 
expect the following: 

1) an instructional program that will help him to acquire a level of coinpe- 
tency which enables h<m to assume instructional responsibilities in a 
classroom: 

2) that the instructional program is a direct outgrowth of identified compe- 
tencies; 

3) that the competencies he )s to attain are explicitly and publicly stated; 

4) that contmuous feedback about his progress will be provided; 

5) that there will be sufficient opportunity to make progress while involved 
in the program; 

6) that the assessment procedure will have publicly stated conditions of 
performance and designated levels of mastery; 

^) that the assessment procedure will include a pre* assessment which 
will allow him to demonstrate his level of performance prior to his in- 
volvement (enrollment) m any component of the instructional program; 

B) that a record of performance will be continuously available to him in a 
profile form; and, 

9) that the standards of achievement to receive recommendations for 
certification are established based on the preceding criteria. 
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Tabit 21,2 

REPRESENTATIVE STUDENT TEACHER EVALUATION FORM 



ASSESSMENT STANDARDS CODE 

C-Category 

D-Definition 
SR'Subjective Requirement 
SC'SubjQctive Criteria 
PR-Performance Requirement 



1 PREASSESSMENT OF PUPiL SKILL AND TECHNIQUES BY 
TEACHER. LEARNER 

Individualization of learning strategies according to pupti needs 
Items Provision for meeting individual differences (SR) 



2. PLANNING INSTRUCTIONAL OBJECTIVES AND STRATEGIES 
Lesson design, umt design 

Items The student teacher develops lessons with consideration 
for pupil pace (SO 



Pacing, variety 
of technique, use 
of unexpectecf 
material 



Alters pace of 
teaching to meet 
pupil needs 



_j 



Not aware 
of student 
response to 
lesson 



3 SETTING UP PROCEDURAL ROUTINES FOR INSTRUCTION 

V Selecting and organizing procedures, equipment, and facilities 
Items Selection, use. and care of room, equipment, and mate* 
nais (SR) 

Regulates physical conditions of the room; lights, head 
ventilation (PR) 

2 Routine duties 

Items. Accurate reports handed m on time (PR) 



5. MAINTENANCE OF A POSITIVE LEARNING ENVIRONMENT 

1 Classroom control direction and procedure 
Items Classroom direction and procedure (C) 

2 Disciplining disruptive pupils 

Items Adequate classroom control (SR) 

Just and effective m handling disruptive students (SR) 

3 Positive learning approach 
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Items. Pupd praise {C) 
4. Pupil participation in learning activity 

Items: Ability to involve all pupils activity (SR) 

6. CONDUCTING LEARNING EXPERIENCES 

1 . Motivation and presentation 
Items Motivational devices {C) 

Dramatic motivations (SR) 

Imaginative motivations (SR) 

introduction and stimulation of activity (SR) 

2. Questioning and ansv^ermg skills 
Items. Questioning (C) 

3. Instructional materials, techniques, visual aids, audio-visual skills 
Items. Use of two visual aids in teaching a lesson (PR) 

Effective use of blackboard (SR) 

Sufficient use of audio-visual equipment (SR) 



7 KNOWLEDGE OF CONTENT AREA, PROCEDURAL SKILLS USED 
DURING INSTRUCTION 

1 . Communication techniques 
Items: Voice and speech (C) 

Student teacher vocabulary (C) 
Speaking distinctly with expression and quality (SR) 
Pleasant, well modulated voice pattern (SR) 
Command of English language* good vocabulary (SR) 

2. Utilization, comprehension of principles m content area, basic 
concepts 

Items. Subject matter comprehension, the understanding of 
concepts through teaching (D) 
Principles m subject matter (C) 
Knowledge of subject (SR) 

Student teacher knowledge of subject matter to be intro- 
duced (SC) 



Resourceful 
imaginative 
use of pupil 
background to 
develop content 



Works into teacher 
guide, selecttveiy 
adds own materials 



Non selective, 
not usually 
prepared 



8. ASSESSMENT>FEEDBACK*REMEDIAL HELP 

Individual and group evaluation of pupils 
Items: Effective use of evaluative devices (SR) 
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ASSESSMENT OF CONTINUING PROFESSIONAL DEVELOPMENT 
Openness to critical comments by others 
Items Acceptance of constructive criticism (SC) 

J I 



Seeks suggestions, Takes suggesttons Fails to follow 
always wilhng \o to & hmite0 dogree suggesttons 

accept suggestions 



PPOFESSIOMAL CHARACTERISTICS, INTERESTS OF A TEACl i.f< 
1 Role awareness and ethics 

Items Dedication to teaching (SR) 
initfattve (SR| 

Professional ethics m evidence <SR) 
Judgment and tact (SR) 
Dependability, prompt (SR) 
? Personal and professional characteristics 
Items Poise, self-confidence (SR) 
Pleasing appearance (SR) 
Intellectual curiosity (SR) 
Good health, vitality (SR) 
Good posture (SR) 
3 Affective capacities Enthusiasm, desire, sympathy 
Items Openmmdedness (SR) 

.ilthy pupil'teacher relations (SR) 
Emotional stability (SR) 
Warmth of personality (SR) 
Sensitivity, rapport (SR) 
Sense of humor (SR) 
Patience, self-control (SR) 
Sociability, friendliness (SR) 
Spontaneity, enthusiasm (SR) 
General quality of relations with children. (SC) 

Ability to make and maintain contact with children, 
empathy; ability to relate m imaginative ways; accep- 
tance of and respect for childrens' work; understand- 
ing of their need for challenge, reactions to negative 
behavior (SC) 



Relationships are Relationships are Poor relations 
good I n all or most adequate but 
aspects uneven 
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Tabl«21.3 

EDUCATIONAL ASSESSMENT SYSTEMS CORPORATION S 
ANALYSIS SYSTEM 

TEACMER-LEARNER PRE-lNSTRUCTlONAL FUNCTIONS AREA 

1. PREASSESSMENT OF PUPIL SKILL AND TECHNIQUES BY 
TEACMER-LEARNER CATEGORY 

1 . Development and use of pupil protiies-achievement ratings 

2. Individualization of learning strategies according to pupi> needs 
3 Diagnosing pupii problems and needs prior to instruction 

2 PLANNING INSTRUCTION OBJECTIVES AND STRATEGIES 
CATEGORY 

1 Lesson design, unit design 

2. Establishing oPjectives 

3. Establishing evaluative criteria 

4. Establishing requirements 

5 Making and selecting tests 

6 Sequencing instructional activities 

3 SETTING UP PROCEDURAL ROUTINES FOR INSTRUCTION 
CATEGORY 

1. Selecting and organizing procedures* equipnnent, and facilities 

2. Routine duties 

3 Observation of instruction 

4 OCCUPATIONAL RESPONSIBILITIES CATEGORY 

1 Research and projects development of instructional resources 

2 Scoring tests and grading 

3. Providing resources and services —bulletin boards, etc 

4 Co-operation with colleagues 

5 Fulfills responsibilities 

6 Providing teache. assisiance 
7. Using consultant personnel 

TEACHER-LEARNER INSTRUCTIONAL FUNCTIONS AREA 

5 MAINTENANCE OF A POSITIVE LEARNING ENVIRONMENT 
CATEGORY 

1 Classroom control direction and procedure 

2 Disciplining disruptive studt^nts 
3. Positive learning approach 

4 Pupil participation in learning activity 

5. Individual work by pupils 

6. Monitoring of pupil interest 

7. Providing pupil assistance 

6 CONDUCTING LEARNING EXPERIENCES CATEGORY 
1 . Motivation and presentation 
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2. Teacher lecture and demonstration techniques 

3 Quc->{ :>a ng and answer mg skills 

4. in^^riic^i.oai materials, visual aides, audio-visual usage skills 

5 Supei v.aing pupii skill practice 

6. Interactive problem solving, critical thinking, and discussions 

7. Relational learning experiences 

a. Summary and conclusion: reinforcement and review 

7. KNOWLEDGE OF CONTENT AREA AND PROCEDURAL SKILLS 
USED DURING INSTRUCTION CATEGORY 

1 Accuracy of information 

2. Adequacy m employment of procedural skills 

3. Communication techniques 

4. Ut:!:zation and comprehension of principles in content area. 

understanding basic concepts 

5. Provision of application procedures necessary for pupil problem 

solving 

6. Understanding or applying educational philosophy 

8. ASSESSMENT-FEEDBACK.REMEDIAL HELP CATEGORY 

1. Assessment: pupil product. proctf:»s, and knowledge for diag- 

nostic purposes after instruction 

2. i iidividual and group evaluation of pupil work 

3 Feedback and analysis of pupil performance 

4. Remedia' help based on analysis of pupil performance 

5 Re-assessment of pupil learning, evaluative instruments, and 
lesson planning based on feedback and remedia; ;«oip 

TEACHER^LEARNER PROFESSIONAL PROFILE AREA 

9 ASSESSMENT OF CONTINUING PROFESSIONAL DEVELOPMENT 
CATEGORY 

1. Preassessment of performance capacities of teacher learner 

2. Self*analysis and self-evaluation by teacher learner 

3. Openness to critical comments by others 

4 Capacity to identify means of improvement 

5. Pupil learning resu'*« ^enlevement and gain 

6. Puoil reaction to teacher learner 

7 Demonstrating ability to progress as teacher learner 
10 PROFESSIONAL CHARACTERISTICS AND INTERESTS OF A 
TEACHER CATEGORY 

1 . Role awareness and ethics 

2. Personal and professional characteristics 

3. Affective capacities enthusiasm, desire, sympathy 

4. Personal involvement in content area activity 

5. Teacher interests; extra-curricular activity, community activity^ 
etc. 
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A NEW STYLE CERTIFICATION INSTITUTE: 
DATA BANK TRAINING INSTITUTE* 

Introduction 

The purpose of this Data Bank is to collect cntical kinds of information at 
a low cost, which will be of maximum value for providing pertinent infor- 
mation to the New York State Education Department, school districts, and 
colleges about the performance and effect of teacher education programs 
on pupil learning in the public schools. Two types of information have been 
designated as of utmost importance. The first is the final evaluation forms 
used by super v:sors to assess teacher learner's performance m the class- 
room The second is an observation by trained observers of the range of 
these teacher learners' education activities during their training program. 
Using this information, the Data Bank charts an institutional profile of the 
training program's means of assessing student teachers and of their actual 
activities as learners in the classroom. This profile is used to review, evalu- 
ate, and formulate instructional programs in ways which increase the deci- 
sion-making responsibilities and accountability of associated administrators, 
faculty, and students. The purpose is to increase the range of learning 
opportunities provided to the teachers-in-training. and simultaneously to 
provide a widening range of learning opportunities for public school pupils. 
The Data Bank System, through a continuous feedback process, allows the 
training staff at the college and public school level to identify gap areas in 
instruc^^n and to concentrate in the field on filling them. Once the gap is 
identifieu. instructional resources are developed which will enable the 
teacher learner to provide pupil instruction in this neglected area. The for- 
mulation of instruction that is adequate in meeting Data Bank standards 
requires two steps. 

(1) That the means of assessment is stated prior to instruction, and is 
formulated m terms which are observable: 

(2) That instruction is provided which fulfills the requirements of these 
means of assessment. Th's provides for the educational rights of the 
teacher learner as adopted by the Advisory Board of the Regents of 
the State of New York. 

Preassessment of Local Institutional Profile 

A distinguishing feature of our program is that there is a prea&sessment 
profile made of the participating institutions' educational program. This is 
followed by mservice training based on special needs of the participating 
institution, in order to provide for immediate positive actions in areas of 



•The Data bank for me state of New YO'h r>,v,s,on of Teacher Education and Certification 
was de&igned by the Educational Assessrr Systems Corporation. 1432 Amnerst Street. 
Buffalo. New York. 14216 

Thit System iS one o* se^erol designs now being considered for adoption by the New York 
Stoto Education Deportment 
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need When desired, a tollowup profile is provided to determine areas of 
gam as a result of the work done. This pattern »s the model established by 
the Data Bank tor the certification of programs in the State of New York 

The profile w.H consist of a 2-day -isit to the participating institutions 
consisting of: 

(1) Sixteen to twenty 30>minute observations made by the EAS staff of 
the institutions, paraprofessionals. and student teachers 

(2) A collection of information concerning: 

a. Student teaching and other teacher learning assessment instru- 
ments. 

b. Task descriptions of teacher learners' jobs, plus role descrip- 
tions, method course outlines, all available instructional mod- 
ules, and evaluative procedures. 

c Fifteen-mmute interviews with 10 student teachers, four co- 
operating teachers, four method instructors, four supervisors, 
and administrators to determine a picture of the institutions* 
governance operations and advisory board, curriculum proce- 
Oures. program sequence, and management systems for pro- 
viding feedback on program effectiveness. 
The material collected here is necessary m order to provide highly relevant 
kinds of training experiences which reault m immediate program improve- 
ments. 

After this preassessment visit and development of the institutions' profile 
the necessary resource material is organized as follows for the participating 
institutions* particular needs. 



Assessment of Teacher Learners 

Training in assessment techniques for teacher learners will take between 
4 and 5 hours. It will consist of a general session on theory concerning the 
relationship of the --Educational Rights of Students" to the means of assess- 
ing their performance. There are two major contents to be mastered. The 
first concerns the statement of performance criteria for assessment and a 
related report on major errors made by 89 institutions on their student 
teacher evaluation forms. Tiie second concerns the necessary range of 
assessment items if a full training program is to provide an adequate base 
for pupil and teacher learning. Exercises in both content skills are provided, 
including a review. 

(1) Theory, This section will be the introduction of the "Educational 
Rights of Students. - This will be followed by an analysis showing how means 
of assessment must be related to means of inbtruction. if training programs 
are to be effective. Specific requirements essential to relating means of 
assessment to means of instruction are given so that job responsibilities of 
the teacher learner are specified and training in them is provided. Without 
meeting the .e two sets of requirements, no program can operate effectively. 

(2) Statement of Performance Criteria. This area will specify mistakes 
frequency made by educational institutions in the statement of performance 
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assessments tor student teachers These are the "Thou Shalt Not" exer- 
cises. This section ends with the analysis of the participating institutrons* 
own evaluation forms. 

(3) Range of Assessments. This area includes a report on the gap areas 
in student teacher training programs of 89 institutions in New York State. 
Also included will be a statement and definition of the 1 1 Data Bank cate- 
gortes needed for assessment. The Data Bank categories fall into three 
major groups: (1) prior to instruction; (2) during instruction; and (3) in 
addition to instruction. There are 55 total items. 41 of which relate to class- 
room observation. An exercise will then be given in which participants will 
match statements of assessment with the Data Bank categories. Another 
exercise will follow, In which the participants will sort their own institutional 
items for assessment for student teachers under the Data Bank categories. 
The results from these exercises will be used to identify gap areas in their 
own institutional student teacher assessment forms. 

(4) FiUing Gap Areas. In the prior two exercises, the gap areas have 
been identified and the requirements for stating performance criteria clari- 
fied. This meeting deals with the formulation of two necessary forms of 
assessment: convergent types of assessment for which pass/fail types of 
requirements can be stated; and divergent types of assessment problems 
which require the formulation of criteria. Success in formulation of both 
types of means of assessment depend upon identifying specifically the func- 
tion to be performed by the learner or the elements of the problem to be 
solved, which distinguish between varying degrees of achievement. This is 
a critical skill because it determines the requirements of instruction. The 
final exercise in this session win involve the participant in stating task de- 
scriptions for criteria. Included in this exercise is the identification of related 
job specifications for this task and the development of proficiency and com- 
petency tests for determining if the job has been done effectively by the 
teacher learner (80 min. ) 

(5) Anonymous Evaluation of the Session by the Participants. 

Introduction to the Observation System 

Training m the observation system will take between 4 and 5 hours. It 
consists of a lecture demonstration and video tape exercises of actu il class- 
room situations to show how the observation system is used. The training 
will also show how to make an analysis of the observation findings. This 
exercise informs the observer concerning how the observation system pro- 
vides a basis for necessary decision making regarding the implementation 
of pupil and teacher learning. The observation system records 41 discrete 
kinds of ctastioom activities and skills. They aie grouped into nine sections: 

(1) Preassessment. Determines participants' prior knowledge of oo- 
servable teacher tasks. Group analysis of the preassessment test will follow 
for review. 

(2) Goais and f^cans of Presentation. Explains the objectives, require- 
ments, and criteria of the observation training session. This will be the 
means by which the participants will evaluate the success of the workshop. 

(3) Presentation. Lecture demonstration by experienced staff identi- 
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fymg the 41 items on the observation form. A videotape example of each 
item will be used to supplement the definition. 

(4) PracticB. Consists of two different observation exercises, viewed on 
video tapes from two different classroom and content areas. Participants 
will use the observation form for the first time, in order to familiarize them- 
selves With the pattern of using the form. These exercises provide for the 
recognition of the same item m different content areas. (30 min.) 

(5) Assessment as Anafysis. Consists of simulated field training. The 
participants will bo broken down into two groups. Each participant in the 
group will observe four different video tapes of actual classroom situations 
using the observation form as they would in the field. The tapes will be run 
continuously. ^gO^i^, 

(6) Evatuation and Feedback. Using the observation forms done by the 
participants in field training, a reliability check for each tape will bo made. 
Items Will be corrected, and uniformly missed items will be replayed and 
reviewed. • 

r. a (45 mm.) 

(7) flecommencfaf/ons and Decision Making. Using one of the partic- 
ipants corrected observation forms for each of the eight tapes, a compos- 
ite form will be made on a transparency and the items totaled. The pattern 
of gaps and strengths will be analyzed. Using the pattern, the participants 
will then be required to write a recommendation to an advisory board as to 
what actions should i>e taken in order to strengthen pupil and teacher learn- 
ing in gap areas. 

One participant will be chosen to act as the person making the recom- 
mendations to the board. The other participants will be assigned various 
roles as board members, such as student teacher, principal, community 
leader, or director. The board will debate the recommendations and deter- 
mine the decisions to be made relating to pupH and teacher learning. The 
following example of an actual memo is the type of decision that can be 
made. , 
a (45 mm.) 

(8) Reassessment. For the final reevaluation the participants will ob- 
serve themselves going over the entire observation system by viewing 
video Clips of themselves as learners during this program. They will use the 
observation form to determine the range of instruction that has been pro- 

ftv A r- t'" mm.) 

(9) Anonymous Evaluation of ttie Session by the Participants. 



Development of Instructional Programs 

The institutional profile has now been established in the areas of assess- 
ment. Each area of assessment must be matched to an area of instruction. 
Some missing areas (gap areas) need to be developed. This session will 
show how to develop and improve existing instructional modules m these 
areas. 

An instructional module must have certain essential components, such 
Q^j^n^^'" °^ ^^^^ (knowledge-lecture) to be performed, with the 
characterSWrWoken dc*n; (2) demonstration showing alternate ways of 
performing that tasK*(3) a on«ppage analysis of the major elements in the 
demonstration: (4) an exercise for the participant; (5) video feedback; 
(6) a retrial; (7) proficiency test with a pass/fail standard establishing that 
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the teacher learner r, ready to perform the taught skill in the field; (8) speciN 
»c requirements or criteria tor competency which will be used by supervisors 
m the field; and (9) an evaluation of the module by the teacher learners to 
be made immediately after the instructional session. 

The cluster of the institutions* modules constitutes the curriculum of the 
training program From this curriculum, a teacher learner profile is devel- 
oped 

(1 ) EMerciSB. The participants will view capsule modules and rate them 
according to the nine module components and related criteria. Volunteer 
participants will do a capsule module on a familiar, everyday subject. These 
will be rated by the other participants and remediation for missing areas will 
be developed with the group 

(2) Practical Exposure. The participants will view two practice video 
clips of modules m gap areas already tested in the f JCid. (30 min. ) 

(3) Peveiopment T^.^ participants wilt break into two groups and each 
will make a mmimodule m a priority area. They will teach their modulo to 
each other and evaluate it for means of improvement. This represents the 
pilot exercise for developing a full-scale module. {75 min.) 

(4) PracUcaf Experience. The participants will act as an advisory board 
in order to designate tasks and necessary work. A framework will be es- 
tablished for maintaining a teacher*learner profile including meir students' 
records from the beginning to the completion of their certification program. 

An Outline of the mslructioridi sequence necessary for achievement can 
then be determined on this profile with respect to its specific elements as a 
curriculum. A timetable for the development of needed elements in the 
instructional program will then be formulated. 

At this point, using all the data from the three training sessions, a new 
institutional profile will be recognized as a new responsibility. <80 min.) 

(6) /Anonymous Evaluation of the Session by the Participants. 
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Implementation of 
CBTE - Viewed as a 
Developmental 
Process* 



Gtne E. Hall 

Innovation adoption in educational institutions is an activity that is often 
approached with httle tndepth calculation and anticipation of potential con- 
sequences It IS all too common to observe an air of casualness that is 
alarming to those repeatedly exposed to change. Often it seems that deci- 
sion makers decide to adopt one of the recently developed complex inno- 
vation bundles as (e.g., individually guided education; competency based 
teacher education! so casually as they would change a textbook series, 
which IS a change that should not always be taken lightly Adoption of inno- 
vations in educational mstituttoiis is not a simple phenomenon or singular 
event. Adoption of educational innovations is a process that generates 
diverse and. m many cases, altogether unexpected outcomes innovation 
adoption can consume much of the energy of the people involved and can 
cost a great deal m terms of resources, dollars, personnel productivity, and. 
not least of al!, time. 

In this paper, an attempt wtii do maoe to share with CBTE program mana- 
gers the experiences, theory, and research findings of the CBAM project 
staff of the Research and Development Center for Teacher Education This 
staff has been actively mvc'ved m studying the CBTE program development 
and adoption processes .n teacher training institutions around the Nation. 
Out of this work and experience a model of innovation adoption" in educa- 
tional institutions has been developed Based on this model, measurement 
procedures and principles for * adoption agents * are being developed and 



• The research described here.n was conducted under contract with the National institute of 
Education The op.mons eipressed are those of the author and do not necessarily reflect the 
position or poi-cy oi the National mst.Tute oi Educat.on and no endorsement by the National 
Institute of Education :»nouid be interred 
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researched Each of Ihose areas of work will be briefly explored in this 
paper Implications for planned change will also be presented as food for 
thought for CBTE program managers 

CBAM Pro|ect Overview 

Researching, planning, and managing innovation adoption m educational 
institutions are the foci of the work of the Concerns Based Adoption Model 
Project of the Research and Development Center tor Teacher Education of 
the University of Texas at Austin In this work, innovation adoption is viewed 
as a complex process rather than as a singular event Phases of this process 
have been identified and described and are being studied. We hypothesize 
that each and all individuals withm an educational institution progress 
through a series of growth steps m developing their capability to effectively 
use an innovation We further hypothesize that adoption agents who assess 
these developmental stages m their clients tmprove their ability to select 
and make personali?ed interventions, thereby accelerating advancement to 
effective and extensive use of the innovation withm the institution. 

A concentrated attempt is being made to conduct empirically based 
studies and at the same time develop procedures and techniques that have 
immediate values for aGwpti<3flwge:^ts and program managers. Although 
much of the early research and development was done in institutions that 
were developing competency based and personalized teacher education 
programs, the findings seem to be applicable to other educational institu- 
tions and other educational innovations. The research thrusts are targeted 
toward several fronts 

1 Identification and description of the phases and steps entailed m 
innovation adoption in educational institutions 

2 Deveiopmeni of assessment methods for predicting user system 
potential for successful adoption of innovations 

3 Development of measurement procedures for assessing and diag- 
nosing the developmental stages and levels of individuals and the 
composite user system 

4 Transfer to and immediate application of the research techniques 
and findings to the activities of adoption agents 

5 Using the work and experiences of adoption agents as a source of 
hypotheses and as a heuristic for learning more about innovation 
adoption m educational institutions 

The remainder of this paper is devoted to brief descriptions of each of these 
research and development activities 

instttutional Phases in the Adoption Process 

Hall (1973). in viewing the history of educational institutions over an ex- 
tended period of time, has identified what appears to be an ebb and flow 
in the intensity of innovative activity. Educational institutions seem to oscil- 
late between extended periods of quiet and relatively shorter periods of 
great activity The extended periods of quiet have been labeled periods of 
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BQuiUbnum. and the shorter, highly active periods have been named perioas 
ot aisequilit>rfym. Th»s does not mean to say that during periods of disequilit). 
rium everything is out of control, but rather that the user system as a whole 
and the individuals withm a are experiencing unusual events, problems, 
imbalance, and increased dynamism However, to many of the institutions' 
members a period of disequilibrium may seem to be a time when things are 
out of control— as if the day-:o-day hum of routine were disrupted with the 
cacophony of builders with jackhammers attempting to build a new order to 
things. In most cases this building means tearmg down some of the old 
structures, changing work patterns, and meeting new challenges. These 
changes will be perceived differently by each person. ^ 

If the attempt at innovation adoption is completely successful, institutions 
are hypothesized to move through six phases of disequilibrium. These 
phases ^^re; 

1 Injection— the idea of the innovation is introduced to members of 
the institution 

2 Examination -~\he innovation receives study, talk, visits, thought, 
planning, research, and committee formation 

3. Preparatior$ "{he time following the commitment to try out the inno- 
vation When materials and resources are organized and pre-use 
training occurs 

4 Samp//ng— first tryout of the innovation on an experimental basis by 
part of the user system 

5. Spraaof— the spread of trial use of the innovation to all potential users 
within the user system 

6. Institutfonafiiation^Xhe innovation comes into regular use by ail 
or nearly all potential users 

As IS represented in Figure 22.1. institutions appear to oscillate between 
periods of equilibrium and periods of disequilibrium. Withm any period of 
disequilibrium, the number of phases and the duration of each phase is 
dependent cn a number of variables, such as the complexity of the innova- 
tion, the users* ability to use similar innovations, the users* level of use of 
previously adopted innovations, the leadership of the adoption agent (s), 
and a host of other variables. Termination of attempted innovations at one or 
another phase of adoption is an all too common occurrence. 



Predicting Potential for Siicceesful Adoption 

Manning (1973) has developed an experimental instrument entitled "The 
Trouble Shooting- Checklist** (TSC) for assessing a teacher training insti- 
tution's potential for successful adoption of two innovations: instructional 
modules and a counseling technique called Personal Assessment Feedback. 
The TSC can be completed by an adoption agent during an initial visa to an 
institution that is considering adoption of an innovation.' 
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Through extensive review of Ihe research literature and intensive inter* 
views with successful change agents and analysis of their responses to 
early forms of the TSC. Manning has developed a set of items that the adop* 
tion agent rates for each of the following categories 

1 Organi2ational structure 

2 Personality and leadership styles of oigani^ation members 

3 Communications 
4. Level of usage 

5 Characteristics of students withm the institutions 
A series of subfactor and factor scores result which make possible the clas- 
sification of an educational institution into one of three categories The 
institution IS classified as being either 

1 An Ideal institution for successful adoption of innovations 

2 A marginally acceptable institution for successful adoption of inno- 
vations 

3 An unacceptable institution for successful adoption of innovations 
Once this classification is made, the TSC manual provides the program 

manager or adoption agent with information about a likely sequence of 
events in the adoption of an innovation in that category of institution. The 
TSC manual also includes information for the adoption agent about the skills 
that are most nkeiy to be effective given the institution's predicted potential 
for successful innovation adoption. These guidelines for adoption agent 
skills are classified under five headings 

1 Leadership Style 

2 General Cognitive Skills 

3. General Communication and interpersonal Skills 

4 Relationships that the Adoption Agent has with the Faculty 

5 Reiii'ionships that the Adoption Agent has with the Administration 
Wallace (1973) has also identified a set of principles for effective change 
agents and elaborated on these. This complementary set of guidelines needs 
to be considered m planning for innovation adoption.^ 

Figure 22.2 is a sample of the TSC Form A for module-ac opting institu- 
tions The subcategory is Organization Structure: the adoption agent is 
asked to select the eight items out of the list that most directly apply to the 
user system being considered This procedure is then repeated for the other 
TSC categories The items are then assigned a weight and the weights are 
summed up to yield a subfactor score in Category A. the items are weighted 
as follows 



1) 


0 


5) 


0 


9) 
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13) 
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24) 


0 



The subscaie score range has been tentatively assigned as follows: 
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Score Ranges 0-4 unacceptable 

5*10 marginally acceptable 
1M6 Ideal 

Research efforts are presently underway to validate these score ranges 
and to refine the TSC interpretation of the data must t}e carefully weighted 
by the CBTE program manager, however, the factors and factor scores 
should provide htm with some added information (although tentative) and 
insight into what the real potential capabilities and inherent risks are in 
facilitating a given institution s adoption of CBTE. 



FHlurt 22.2 

TSC- A' 

(FOR MODULE-ADOPTING INSTITUTIONS) 
SECTION I 

The following TSC categories and items focus on the institution's organi2a- 
tional structure and include characteristics of the faculty and administration 
as they relate to organizational structure. 

CHECK ONLY THE S ITEMS THAT MOST APPLY. 

Category A: Organizafton Structur0 

1 The internal change agent working at this institution appears 

to be incompetent, and his position lacks authority and le* 
sponsibility 

^ 2. There is little state level support or leadership 

3 The group of potential adopters seems to have some commu- 
nication problems with the larger faculty group. 

4. There IS a small group of adopters which has credibility wtth 

a larger faculty group tnat gives feedback. 

5 The potential adopters that do exist have serious communi- 
cation problems with the larger faculty group. 

—-^ — - 6 The internal change agent working at this institution, although 
quite capable, is not in a position of authority. 

7 It IS not yet clear how large the group of adopters will be. 

8. The internal political structure is such that the tenured faculty 

exerts pressure against innovation. 

9 There is an 'intellectual * authority figure tn addition to "line- 
Staff ' authority 

^ 10 The organization has a stable structure with fairly well defined 

roles and established (tunctionah channels of communica- 



* Brad A Manning. Tho Trouble* Shooting Chdcftlisl A Manual to Ard Educalfonal Change 
Agenis in the Prediction of Organt^altonai Change Potent id)" (Auttm Univerftily of Texas. 
Research and Development Canter for Teacher Education 1973) 
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11 

— 12 
13 

14 

15 

16 

..^ 17 



. 18 

19 

20 
21. 

22 

23 
24 



uon 

There is no intellectual * authonty figure---only "line-staff 
authority. 

The source of power lies outside of the institution. 

The fnternal change agent working at this institution is in a 
position of authority and responsibility. 

There is a small group of highly involved adopters who work 
m close proximity. 

There is a small group of adopters appearing to move faster 
and more effectively than would a large group of adopters. 
There \s a small group of adopters who clearly demonstrate 
an ability to effectively communicate with a largei faculty 
group m order to gam Iheir Support 

There are a number of potential adopters, but none who are 
yet fully committed 

Potential adopters are scattered across campus and do not 
have daily contact. 

There is a closed organizational structure. (All activities fi» 
into a predetermined structure ) 

There IS a strict, hierarchical organization. 

The group of adopters has not yet established credibility with 
a larger faculty group but clearly shows potential to do so. 
The organization structure includes the following hierarchy of 
positions president, provost; dean; and department chair- 
man. 

There are no committed adopters or potential adopters 
identifiable 

Those individuals who ha* c e.<pressed interest m the innova- 
tion have low credibility w:th the rest of the faculty and appear 
to be locked into their pos tions. 

CATEGORY UA SCALE SCORE 



Concerns Based Adoption Model (CBAM) 

Hail. Wallace, and Dossett <1973) have developed a model to represent 
the complex process of innovation adoption m educational institutions The 
modei-^the Concerns Based Adoption Model (CBAM)-^is an attempt to 
represent the complex innovation adoption process by clustering the many 
possible variables mto a condensed set of dimensions and interrelationships 
that can be used for study. A model that is as complex as the process repre- 
sents would not be of value to either the researcher or the practitioner The 
CBAM IS based on viewing innovation adoption as a developmental process. 
The CBAM organizes the innovation adoption process around the concepts 
of collaborative linkage, individuals* use of the innovation and individuals' 
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concerns about their use ut the innovation information about use and con* 
cern provide the adoption agent with the basis for selecting personalized 
assistance strategies * The basic dimensions of the CBAM are explored m 
the next few pages 

Description of CBAM 

Coli«t»orativ9 Unkagt 

The CBAM begms with viewing the adopting msttiution as a user system 
composed of individuals, each of whom has his own set of concerns, prob- 
lems, skills, agendas, and needs in combination, these individuals repre- 
sent the institution and its functioning When this user system becomes in- 
volved in adopting an innovation, a resource system that is ejcper t in the use 
of the innovation is normally available to help it develop its capability Some- 
times the resource system is an individual, sometimes it is located with the 
user system, usually, however, it is a formal organization outside the user 
system that forms a linkage with the user system. 

Whatever form the resource system takes, for best results with all com- 
plex innovations the linkage should be a collaborative one based on mutual 
openness m communication and a sharing of resources, investments, out- 
comes, and risks <see Figure 22.3) A one-way association is not likely to 
survive because the receivers will not sustain a commitment to a joint effort 
The CBAM requires that investments be made by both user and resource 
systems, and that both be able to gam from the collaboration in most in- 
stances a cotiBborattve tmkage is established to help the user system de- 
velop an effective use of the innovation as quickly and as easily as possible 
This means that with time the individuals within the user system must be- 
come as knowledgeable about the innovation as are the members of the 
resource system m addition, each individual m his role— whether it be as 
an administrator, faculty member, or student— must develop the skills and 
finesse m usmg the innovation that will optimize the effects of its use. 



Ptgure 22.3 

BASIC REPRESENTATION OF THE SYSTEMS 
OF THE CONCERNS BASED ADOPTION MODEL 
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One prem.se utuJerlymg ttm CBAM is that adoption agents (specialists in 
the use of the innovation and effective catalysts for facilitating change) work 
wtth people in the user system both mdividually and in groups. As a result, 
the CBAM at one level focuses on assessing the current stale of the indi- 
viduals within the user system This assessment then allows the adoption 
agent to focus his interventions so that they respond to the perceived needs 
of the individual users and also relate to their leveis of use of the mnovalion 
at that time The constructs of the CBAM that make this possible are the two 
sets of scales 

1 Levels of use of the innovation 

2 Stages of concer n about the innovation 

in addition, a third and more specific set of hypotheses has to do with the 
relationship of stages of concern to levels of use 
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Levels of Use of the Innovation 

We contend that there are observable differences m how various indi- 
viduals approach and use an innovation. Specifically, it is hypothesized 
that there are identifiable, definable, and measurable levels of use of an 
innovation that range from lack of knowing that the innovation evOn exists 
into an active, sophisticated, and highly effective use of it It is further hy- 
pothesized that improved us© of the innovation by most individuals is a 
developmental process Normally, individuals do not use an innovation for 
the first time, or even the second time, as efficiently and effectively as do 
those Who have been involved wi'h the innovation through four or five cycles 
of use Advanced levels of use are not atta.ned merely by repeated use of 
the innovation, however Experience is essential but not sufficient to insure 
that a given individual will develop the ability to use an innovation effectively. 

An oversimplified but helpful illustration of the level-of-use dimension is 
the process a college instructor goes through when he adopts a new text- 
book for a -ourse he has taught many times. At first he will use the new 
book for reference much mce than he did the old text In preparing class 
presentations and examinations, he will refer to it much more His assign- 
ments are more likely to be specific chapter assignments, and he probably 
Will follow a straightforward progression through the text. His use of the 
innovation is apt to be mechanical, uneven m flow, and closely related to 
the text As this instructor prepares to teach the course a second time using 
the new text, however, he is likely to select a different arrangement of as- 
signments This time, he may assign Chapter 4 first and. perhaps, delete 
Chapter 7 while substituting another reference he thinks will do a better job 
in making these changes, he has progressed beyond a mechanical use of 
the innovation. He has gamed the experience and know-how to be more 
adaptive in his approach, and he more smoothly integrates the use of the 
text into the rest of his instructional activities 

For research, operational definitions and scale points for the levels-of-use 
dimension of the CBAIW have been developed. To illustrate. Table 22.1 con- 
tains sample behaviors foynd at each level. Two subscales are hypothesized 
for the levels-of-use dimension One describes the knowledge level of the 
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user. It hypothesises thai ti.e cognitive level, or amount of information and 
degree of under^andmg an individual user has at)out the innovation, is a 
developmental progression. Assessment of this set of scale points might 
take the form of a pencil-paper achievement test. The other set of scale 
points for level-of-use of the innovation are the action scale points. In the 
CBAM. one hypothesises that there are ot)servable behavioral differences 
in how the innovation is actually used and that advancement to the higher 
levels of use of the innovation is a developmental process Assessment of 
the action level of use requires direct observation of the users while they 
use the innovation. 



Table 22.1 

LEVELS OF USE AND TYPICAL BEHAVIORS 
FOR EACH LEVEL OF USE OF THE INNOVATION 



Level of Use 

Non-Use 

Or lentation 

initial Training 
Mechanical 

Independent 
integrated 

Renewing 



Behavioral Indices of Level 



No action is being taken to learn about new ideas in 
the area of the innovation. 

The user is seeking cut information about the inno- 
vation. 

The user IS preparing to use the innovation. 

The user ts usmg *he innovation in an awkward, poorly 
coordinated manner 

The user is doing a good )Ob with the innovation. 

The user is sharing with others what he i$ learning 
about students from using the innovation. 

The user is seeking out more effective alternatives to 
his established use of the innovation. 



Stages of Concern About tbe Innovation 

A second dimension has to do with the individual user's needs, motiva- 
tions, problems, and requests as he »s becoming expert in using the innova- 
tion in a way that parallels Fuller s (1969) studies of concerns of teachers, 
individuals are hypothesised to have concerns that relate to their potential 
or actual use of an innovation. A set of scale points — Stages of Concern 
About the Innovation— has been defined for this dimension, and it is 
hypothesized that this dimension is also a developmental progression. That 
IS, when the individuals first approach usmg an innovation, their concerns 
will be different from those they will have after they have used it a while. Still 
higher stages of concern will be expressed with subsequent cycles of use 
unless one or more developmental processes become blocked or dormant.^ 
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As With Fuller s theory of concerns of teachers, the CBAM hypothesizes 
that early concerns are much more self-oriented than are later concerns. 
Table 22.2 lists Stages of Concern About the Innovation ranging from Un- 
aware to Renewal with typical expressions of concern. 



Tablft 22.2 

STAGES OF CONCERN AND TYPICAL EXPRESSIONS 
OF CONCERN ABOUT THE INNOVATION 



Stage of Concern 



Unaware 
Awareness 

Exploration 

Early Trial 

Limited 
Impact 

Maximum 
Benefit 

Renewal 



Expressions of Concern 



I don t know anything about it (the innovation). 

I have heard about the innovation, but I don t know 
much about it. 

How much of my time would use of this innovation 
take? 

I seem to be spending all my time in getting material 
ready for students. 

I can now see how this innovation relates to other 
things I am doing. 

I am concerned about relating the effects of this inno- 
vation with what other instructors are doing. 

I am trying a variation m my use of the innovation that 
looks hke It is going to result in even greater effects. 



Relatlonthtp Between SoC and LoU 

It IS hypothesized that concerns are related to use and that it is possible 
for Change agents to infer a great deal about use of the innovation from 
listening to the user s concerns. This relationship is not always a simple one^ 
to-one correspondence, however. Many of us, for example, have known 
golfers who talked a good game ' but whose actual play was rather far over 
par. The alternate imbalance m theory is also possible where the mdividuars 
concerns are of a very low level and he has serious doubts about his abilities 
when, in fact, he has the potential of being outstanding. There are also in- 
stances of individuals who -perform over their heads.** 

An Illustration of these relationships using an educational innovation could 
be schools adopting open concept classrooms. Many communities now have 
school buildings that are open concept and have reputations for having 
exciting, innovative programs. When one visits some of these schools, how- 
ever, he finds that bookcases, chalk boards, easels, and seating are ar- 
ranged m blocks that serve as traditional self-contained classrooms (low 
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use). In another school, where all the walls are gone, the pupils are roving 
aimlessly. No territories have been established; there is excessive confu- 
sion; and the Climate feels tense. This is a school that is probably early in 
its use of open-concept classrooms and where the teachers have high 
stages of concern about sharing their leadership and responsibility for Cur- 
riculum and about remaining nonauthontanan. But, in spite of these high 
concerns, their level of use of open-concept classrooms is low. They are 
confused and uncertain as a result of perhaps attempting too ambitious a 
beginning. 

With the CBAM. It IS hypothesized that there is probably a middle range of 
relationship between concerns and use where successful advancement or 
growth is possible, but if an individuafs stage of concern and level of use 
move too far out of correspondence, then adoption of the innovation is in 
jeopardy. Figure 22.4 is a graphic representation of this set of hypotheses, 
with the area within the envelope representing the hypothesized safe-growth 
area. 



ExtensHy 

The ultimate criterion in any innovation adoption effort is the extent and 
quality of use by each user of the innovation within the user system. The 
leveUof-use dimension of the CBAM contains a set of operationally defined 
scale points that provides behavioral indicators of the quality of use of an 
innovation by each individual within the user system. Innovations are 
adopted by user systems composed of many individuals, it is important to 
have a record of each individual s level of use. Also, a representation of 
the proportion of individuals within the user system that are using an innova- 
tion needs to be made. A descriptive statement that the average user in a 
school is at a mechanical level of use is not as useful as is a picture of the 
present level of use that each individual is demonstrating. An extensity 
profile can be constructed to accomplish this. All faculty, administrators, 
and students can be observed and rated with respect to their levels of use 
of the innovation. When this information is plotted, the resultant graph 
represents the individual level of use and the extent of use of the innova- 
tion within the user system at the time the observations were made. By 
plotting extensity profiles at regular intervals, a visual record can be main- 
tained of the extent and level of use of the innovation. When extensity pro- 
files for different dates are compared, the rate of advancement of innova- 
tion use or its arrest can easily be seen. Figure 22.5 is an example of an ex- 
tensity profile. 



Putting It Together 

The Concerns Based Adoption Model in its entirety is represented in Fig- 
ure 22.6. In operation, there is a collaborative linkage established between 
a user system that is adopting an innovation and a resource system that has 
expertise with the innovation and facilitati <g its adoption. In theory, linkage 
IS accomplished via several communication channels that entail systematic 
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probes of the user system and its personnel to assess each u$er*s stage of 
C0r>cern and level of use about the innovation. Based on this assessment, 
ddcpt on agents should be better able to select and employ personalized 
intervention Strategies. The selected strategies are targeted toward ad- 
vancing use of the innovation while, at the same time, resolving the user s 
concerns or arousing more advanced concerns. Interventions that are 
targeted m this way are most likely to appear as relevant to the user's con- 
cerns and, thereby, are most likely to effect advancement in the level of use 
of the innovation. 



Adoption Strategies 

The actual implementation of CBTE can be approached using one or a 
combination of several different adoption strategies. An adoption strategy 
IS a general game plan or plan of action that is designed to move a user 
system from first awareness of an innovation to high level and extensive use 
of the innovation The adoption strategy selected will greatly influence the 
rate of spread of use of the innovation, and each Strategy has inherent 
characteristics that can lead to the emergence of certain barriers to suc- 
cessful adoption. 

Presently, at least 1 1 auoption strategies that have been employed ex- 
tensively can be identified These are 

1 The Bootstraps Approach. An individual withm the user system or an 
entire user system decides to develop or use a new product. Learning 
how to use It and collecting the necessary resources for using the 
innovation are carried Out by the user system with no Outside support 
or assistance. 

2. The Decree. An individual m a decision making position within the 
user system announces that the innovation will begin to be used as 
of a particular time. In many cases, the decision maker is a person 
who believes in and practices rational decision making. He assumes 
that everyone else will naturally see the obvious advantages to be 
gamed by using the innovation just because it makes sense. In many 
cases, few resources and support systems are allocated to the adop- 
tion process, it is assumed to be a fait accompU with no need for any 
dragged-out development 

3 The God-BlesS' You Approach. An innovation representative or a con- 
sultant works with the user system for a few hours or as long as one 
or two days when the innovation is first adopted. He then leaves with 
the user system left on its own to work out any problems encountered 
and With the expectation that the innovation will be used effectively. 

4. intensive ^retraining, individuals are introduced to the innovation 
through a summer workshop, mservice training session, retreat, or 
Short stay at a training center and then begin using the innovation 
regularly. In some instances, there are occasional followup con- 
ferences or meetings during the first year of use. 

5. Sabbaticai. An individual who will use the innovation takes an ex- 
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tended leave to spend time at a tram.ng center or with an institution 
already using the innovation He then can return to his own institution 
as an advocate of the innovation and facilitate its use 

6 Superstar Strategy The user system imports one or more recognized 
experts as full-time members of the system They may have been 
involved with the original development of the innovation, or may have 
gamed knowledge and exposure from being with an established user 
system. They are expected to implant their skills and help the inno- 
vation become mstitutionaliied and. at the same time, bring national 
visibility to the user system 

7 EMpenmerital Units A small group of individuals from within the user 
system experiments with an innovation on a trial basis. If the inno- 
vation IS successful, it is implemented throughout the user system 
If ft fails, a much smaller percentage of resources are lost than 
would be lost m a blanket adoption. 

8 Blanket Adoption. The innovation is adopted for everyone across the 
user system at the same time. The training that individuals receive 
vanes from none to extensive and prolonged. 

9 Outside Collaboration. The user system hnks with an outside resource 
system for a long-term collaborative relationship. This process allows 
the user system access to the skills, expertise, and other resources 
of the resource system and allows the resource system a field basis 
for evaluating and researching innovation development. 

10. m Pennsylvania Contingent. A new group of people is added to the 
user system, through a change in administration (new superinten- 
dent or dean) The new person brings along several associates from 
his past who work together to bring about Change. 

1 1 Goocf Time Workshops. This nonadopnon strategy is extensively used 
in school systems where a set number of inservice days are sched- 
uled. There IS no real commitment to an innovation or its adoption. 
Rather, the goal is to entertain the teachers for the duration of the 
workshop and have them leaving reporting that it was fun. 

If the CBTE program manager consciously plans for and carefully considers 
the advantages and disadvantages of the adoption strategy selected he will 
be better able to handle problems and facilitate CBTE program development 
and adoption. 



Summary 

This brief survey of the various research and descriptive works that are 
underway m the UTR&D/CBAM Project are offered not as final solutions, 
but as some f<rst attempts to provide program managers involved with the 
adoption of CBTE programs and other educational innovations with some 
hooks and handles to use m planning for and managing the innovation 
adoption process in educational institutions. Having a strong conceptual 
development and designing effective operational procedures are essential 
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ingredients tor a CBTE program However, these conceptual ingredients 
only provide the user system with the package. Putting its contents to use, 
that is, adopting the innovation, is an equally complicated and essential 
process. 



POOTNOTES 

* G E Hall. Phases m the Adoption of Educational innovations m T«ach«f Tf«ining insti- 
tution^ t Austin University ol Texas. Research and Development Center for Teacher Educa- 
tion. 19731 

' B A Manning The Trouble-Shooting' ChecMiist A Manual to Aio Educational Change 
Agents m the Prediction ol Organisational Change Potential' (Austm University ot Texas, 
Research and Development Center lor Teacher Education. 1973) 

^ R C Wallace. Jr . Each Mis Own Man The Role ol Adopt'on Agents m the implementa- 
tion ot Personal i2ed Teacher Education' (Austin University of Texas. Research and Develop- 
ment Center tor Teacher Education. 1973) 

•G E Hdii. et .ii A Developmental Conceptualization ot the Adoption Process withm 
Educational institutions (Austin University of Texas. Research and Development Center for 
Teacher Education. 1973} 

* P F Fuller. Concerns of Teachers A Developmental Conceptualisation.' Am9rican Edu- 
csition^i R99€arch Journat. (1969 1. 207-26 
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